Deep learning

• This will cover key theory aspects
  • Neurons and Activation Functions
  • Cost Functions
  • Gradient Descent
  • Backpropagation

1. Introduction to the Perceptron

• Before we launch straight into neural networks, we need to understand the individual components first, such as a single “neuron”.
• Artificial Neural Networks (ANN) actually have a basis in biology!
• Let’s see how we can attempt to mimic biological neurons with an artificial neuron, known as a perceptron!
• The biological neuron:
• Dendrites树状突（가지돌기)
• The artificial neuron also has inputs and outputs!
  • This simple model is known as a perceptron.
  • We have two inputs and an output
  • Inputs will be values of features
  • Inputs are multiplied by a weight
  • Weights initially start off as random
  • Inputs are now multiplied by weights
  • Then these results are passed to an activation function
    • Many activation functions to choose from, we’ll cover this in more detail later!
  • If sum of inputs is positive return 1, if sum is negative output 0.
  • In this case 6-4=2 so the activation function returns 1.
  • There is a possible issue. What if the original inputs started off as zero?
    • Then any weight multiplied by the input would still result in zero!
  • We fix this by adding in a bias term, in this case we choose 1.
• So what does this look like mathematically?
• Let’s quickly think about how we can represent this perceptron model mathematically:
• Once we have many perceptrons in a network we’ll see how we can easily extend this to a matrix form
• Review
  • Biological Neuron
  • Perceptron Model
  • Mathematical Representation

2. Introduction to Neural Networks

• We’ve seen how a single perceptron behaves, now let’s expand this conceptto the idea of a neural network
• Let’s see how to connect many perceptrons together and then how to represent this mathematically!
• Multiple Perceptrons Network
• Input Layer. 2 hidden layers. Output Layer
• Input Layer
  • Real values from the data
• Hidden Layers
  • Layers in between input and output
  • 3 or more layers is “deep network”
• Output Layer
  • Final estimate of the output
• As you go forwards through more layers, the level of abstraction increases
• Let’s now discuss the activation function in a little more detail!
• Previously our activation function was just a simple function that output 0 or 1.
• This is a pretty dramatic function, since small changes aren’t reflected
• It would be nice if we could have a more dynamic function, for example the red line!
• Lucky for us, this is the sigmoid function
• Changing the activation function used can be beneficial depending on the task!
• Let’s discuss a few more activation functions that we’ll encounter!
• Hyperbolic Tangent: tanh(z)
• Rectified Linear Unit (ReLU): This is actually a relatively simple function: max(0,z)
• ReLu and tanh tend to have the best performance, so we will focus on these two.
• Deep Learning libraries have these built in for us, so we don’t need to worry about having to implement them manually!
• As we continue on, we’ll also talk about some more state of the art activation functions.
• Up next, we’ll discuss cost functions, which will allow us to measure how well these neurons are performing!

3. Cost Functions

• Let’s now explore how we can evaluate performance of a neuron!
• We can use a cost function to measure how far off we are from the expected value.
• We’ll use the following variables:
  • y to represent the true value
  • a to represent neuron’s prediction
• In terms of weights and bias:
  • w*x + b = z
  • Pass z into activation function σ(z) = a
• Quadratic Cost
  • C = Σ(y-a)2 / n
• We can see that larger errors are more prominent(重要的) due to the squaring（原型）.
• Unfortunately this calculation can cause a slowdown in our learning speed.
• Cross Entropy（比Quadratic Cost速度更快)
  • C = (-1/n) Σ (y⋅ln(a) + (1-y)⋅ln(1-a)
  • This cost function allows for faster learning.
• The larger the difference, the faster the neuron can learn.
• We now have 2 key aspects of learning with neural networks
  • the neurons with their activation function and the cost function.
• We’re still missing a key step, actually “learning”!
• We need to figure out how we can use our neurons and the measurement of error (our cost function) and then attempt to correct our prediction, in other words, “learn”!

4. Gradient Descent and Backpropagation

• Gradient descent is an optimization algorithm for finding the minimum of a function.
• To find a local minimum, we take steps proportional to the negative of the gradient.
• Gradient Descent (in 1 dimension)
• Visually we can see what parameter value to choose to minimize our Cost
• Finding this minimum is simple for 1 dimension, but our cases will have many more parameters, meaning we’ll need to use the built-in linear algebra that our Deep Learning library will provide!
• Using gradient descent we can figure out the best parameters for minimizing our cost
  • for example, finding the best values for the weights of the neuron inputs.
• We now just have one issue to solve, how can we quickly adjust the optimal parameters or weights across our entire network?
• This is where backpropagation comes in!
• Backpropagation is used to calculate the error contribution of each neuron after a batch of data is processed.
• It relies heavily on the chain rule to go back through the network and calculate these errors.
• Backpropagation works by calculating the error at the output and then distributes back through the network layers.
• It requires a known desired output for each input value (supervised learning)
• The implementation of backpropagation will be further clarified when we dive into the math example!

5. Manual Neural Network Operation

• Operation Class
  • Input Nodes
  • Output Nodes
  • Global Default Graph Variable
  • Compute
    • Overwritten by extended classes
• Graph - A global variable

6. Manual Neural Network Variables, Placeholders, and Graphs

• Placeholder - An “empty” node that needs a value to be provided to compute output(빈껍데기 이며 아웃풋 계산을 위해 값이 들어가야된다.)
• Variables - Changeable parameter of Graph
• Graph - Global Variable connecting variables and placeholders to operations (오퍼레이션 만나게 해주는 브로커 역할)

7. Manual Neural Network Session

• Now that the Graph has all the nodes, we need to execute all the operations within a Session.
• We’ll use a PostOrder Tree Traversal to make sure we execute the nodes in the correct order.
  • y = mx + b
  • y = -1x + 5
  • Remember that both y and x are features!
  • Feat2 = -1*Feat1 + 5
  • Feat2 + Feat1 - 5 = 0
  • FeatMatrix[ 1, 1] - 5 = 0
• manually build out a neural network that mimics the TensorFlow API. This will greatly help your understanding when working with the real TensorFlow!

Quick note on Super() and OOP

class SimpleClass():

    def __init__(self,str_input):
        print("SIMPLE"+str_input)

class ExtendedClass(SimpleClass):

    def __init__(self):
        print('EXTENDED')

s = ExtendedClass() # EXTENDED
# self = s

class ExtendedClass(SimpleClass):

    def __init__(self):

        super().__init__(" My String")
        #super : grab whatever class im inheriting from  (simpleclass
        #종속 클래스에 넘어간다.
        print('EXTENDED')

s = ExtendedClass()
#SIMPLE My String
#EXTENDED

# self = s

Operation

class Operation():
    """
    An Operation is a node in a "Graph". TensorFlow will also use this concept of a Graph(Global Variable.

    This Operation class will be inherited by other classes that actually compute the specific
    operation, such as adding or matrix multiplication.
    """

    def __init__(self, input_nodes = []):
        """
        Intialize an Operation
        """
        self.input_nodes = input_nodes # The list of input nodes
        self.output_nodes = [] # List of nodes consuming this node's output
        #placeholde
        # For every node in the input, we append this operation (self) to the list of
        # the consumers of the input nodes
        for node in input_nodes:
            node.output_nodes.append(self)
        # self which is essentially just a reference to this current operation
        # graph
        # There will be a global default graph (TensorFlow works this way)
        # We will then append this particular operation
        # Append this operation to the list of operations in the currently active default graph
        _default_graph.operations.append(self)

    def compute(self):
        """
        This is a placeholder function. It will be overwritten by the actual specific operation
        that inherits from this class.

        """

        pass

Example Operations(addition)

class add(Operation):

    def __init__(self, x, y):
        #x,y is the our input node to operation

        super().__init__([x, y])
        #super chain 같은것이다

    def compute(self, x_var, y_var):

        self.inputs = [x_var, y_var]
        return x_var + y_var

Example Operations(Multiplication)

class add(Operation):

    def __init__(self, x, y):
        #x,y is the our input node to operation

        super().__init__([x, y])
        #super chain 같은것이다

    def compute(self, x_var, y_var):

        self.inputs = [x_var, y_var]
        return x_var * y_var

Example Operations(Matrix Multiplication)

class matmul(Operation):

    def __init__(self, a, b):

        super().__init__([a, b])

    def compute(self, a_mat, b_mat):

        self.inputs = [a_mat, b_mat]
        return a_mat.dot(b_mat)

PlaceHolder (변수를 받는 곳)

class Placeholder():
    """
    A placeholder is a node that needs to be provided a value for computing the output in the Graph.
    """

    def __init__(self):

        self.output_nodes = []

        _default_graph.placeholders.append(self)
        #going to grab that global variable from the default graph

Variables

class Variable():
    """
    This variable is a changeable parameter of the Graph.
    """

    def __init__(self, initial_value = None):

        self.value = initial_value
        self.output_nodes = []


        _default_graph.variables.append(self)

Graph

class Graph():


    def __init__(self):

        self.operations = []
        self.placeholders = []
        self.variables = []

    def set_as_default(self):
        """
        Sets this Graph instance as the Global Default Graph
        """
        global _default_graph
        _default_graph = self
        # that's going to allow us to access it inside this place holder insde this calss

Basic Graph

• z = Ax + b
• with A=10 and b=1
• z = 10x + 1
• Just need a placeholder for x and then once x is filled in we can solved it!

g = Graph()
g.set_as_default()
#none .
#_default_grapht is the grobal graph
A = Variable(10)
b = Variable(1)
# Will be filled out later
x = Placeholder()
y = multiply(A,x)
z = add(y,b)

Session(help to execute all the operations)

import numpy as np
def traverse_postorder(operation):
    """
    PostOrder Traversal of Nodes. Basically makes sure computations are done in
    the correct order (Ax first , then Ax + b). Feel free to copy and paste this code.
    It is not super important for understanding the basic fundamentals of deep learning.
    """

    nodes_postorder = []
    def recurse(node):
        #递归
        if isinstance(node, Operation):
            for input_node in node.input_nodes:
                recurse(input_node)
        nodes_postorder.append(node)

    recurse(operation)
    return nodes_postorder

class Session:

    def run(self, operation, feed_dict = {}):
        """
          operation: The operation to compute
          feed_dict: Dictionary mapping placeholders to input values (the data)# mapping data in placeholder  
        """

        # Puts nodes in correct order
        nodes_postorder = traverse_postorder(operation)

        for node in nodes_postorder:

            if type(node) == Placeholder:

                node.output = feed_dict[node]

            elif type(node) == Variable:

                node.output = node.value

            else: # Operation

                node.inputs = [input_node.output for input_node in node.input_nodes]


                node.output = node.compute(*node.inputs)
                #* : just wait for us to provide these inputs without knowing how many inputs we may have throughout the operation
                #so technically we don't really know the size of this list so we pass it into computer

            # Convert lists to numpy arrays
            if type(node.output) == list:
                node.output = np.array(node.output)

        # Return the requested node value
        return operation.output

sess = Session()
result = sess.run(operation=z,feed_dict={x:10})
result # 101
# 10*10+1

g = Graph()

g.set_as_default()

A = Variable([[10,20],[30,40]])
b = Variable([1,1])

x = Placeholder()

y = matmul(A,x)

z = add(y,b)
sess = Session()
result = sess.run(operation=z,feed_dict={x:10})
result
# array([[101, 201],
       # [301, 401]])

Activation Function

import matplotlib.pyplot as plt
%matplotlib inline

def sigmoid(z):
    return 1/(1+np.exp(-z))

sample_z = np.linspace(-10,10,100)
sample_a = sigmoid(sample_z)

plt.plot(sample_z,sample_a)

Sigmoid as an Operation

class Sigmoid(Operation):


    def __init__(self, z):

        # a is the input node
        super().__init__([z])

    def compute(self, z_val):

        return 1/(1+np.exp(-z_val))

Classification Example

from sklearn.datasets import make_blobs
#make_blob(binary Large object): 이미지 사운드 비디오 같은 멀티미디어 데이터를 다룰떄 사용 된다.
data = make_blobs(n_samples = 50,n_features=2,centers=2,random_state=75)
#center is how many blob we going to make
# value in array is the n_feature

data

(array([[  7.3402781 ,   9.36149154],
        [  9.13332743,   8.74906102],
        [  1.99243535,  -8.85885722],
        [  7.38443759,   7.72520389],
        [  7.97613887,   8.80878209],
        [  7.76974352,   9.50899462],
        [  8.3186688 ,  10.1026025 ],
        ....
array([1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1,
        0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0,
        0, 0, 0, 1]))

features = data[0]
plt.scatter(features[:,0],features[:,1])

labels = data[1]
plt.scatter(features[:,0],features[:,1],c=labels,cmap='coolwarm')

# DRAW A LINE THAT SEPERATES CLASSES
x = np.linspace(0,11,10)
y = -x + 5
plt.scatter(features[:,0],features[:,1],c=labels,cmap='coolwarm')
# to make color , c=labels, cmp='coolwarm'
plt.plot(x,y)

Defining the Perception

• 𝑦=𝑚𝑥+𝑏
• 𝑦=−𝑥+5
• 𝑓1=𝑚𝑓2+𝑏,𝑚=1
• 𝑓1=−𝑓2+5
• 𝑓1+𝑓2−5=0

Convert to a Matrix Representation of Features

• $𝑤^𝑇$𝑥+𝑏=0
• (1,1)𝑓−5=0

Example Point

• Let’s say we have the point f1=2 , f2=2 otherwise stated as (8,10). Then we have:
  • (1,1)$(8 10)^T$+5=

np.array([1, 1]).dot(np.array([[8],[10]])) - 5 #array([13])
np.array([1,1]).dot(np.array([[4],[-10]])) - 5 #array([-11])

Using an Example Session Graph

g = Graph()
g.set_as_default()
x = Placeholder()
w = Variable([1,1])
b = Variable(-5)
z = add(matmul(w,x),b)
a = Sigmoid(z)
sess = Session()
sess.run(operation=a,feed_dict={x:[8,10]})
# graph output
# 0.99999773967570205

sess.run(operation=a,feed_dict={x:[0,-10]})
# 3.0590222692562472e-07

Reference

Pieriandata

Review

• Multi-armed bandit Review(Bayesian machine learning: A/B Testing)
• Explore-exploit dilemma
• 4 Algorithms:
  • Epsilon-greedy
  • Optimistic initial Value
  • UCB1
  • Thompson Sampling
• Basic definitions in RL
• Tic-Tac-Toe
• MDPs
• Policies state-value functions, action-value funtions
• Return
• 3 methods:
  • Dynamic Programming(direct application of bellman’s Equation)
    • Policy iteration, Value iteration
  • Monte Carlo
    • Learning from experience
    • Not fully online
  • Temporal Difference Learning
    • Fully online with bootstrapping
    • Also learn from experience
• Approximation Methods
• Tabular methods can be infeasible for large state spaces
  • action value funtion을 Q-table로 작성하여 푸는 방법
• differential models(feature engineering)

  1. Review of MDPs

• Markov decision Processes
• MDPs a collection of 5 things:
  • set of all states
  • set of all actions
  • set of all rewards
  • state transition probabilities
  • Discount factor(gamma)
• States
  • State represents what the sensors of our agent measure from the environment
  • In GridWorld, that would be our position on the board
  • in tic-tac-toe: specific configuration of pieces on the board
  • Video game: pixel on the screen
  • Maybe also : # of lives we have left, health, etc
  • For an AI to be as human as possible maybe we should require it to learn that only from pixels on the screen
• Actions
  • Anything the agent can do while in a state
  • tic-tac-toe: placing a piece on the board
  • Video game: moving up/down/left/right, pressing an action button
• Rewards
  • Agent receives a reward at every time step
  • reward are real-valued
  • goal of agent is to maximize total future reward
  • careful to define rewards the right way
  • Ex. Robot trying to solve a maze receives a reward of 0 at every step and 1 for solving the maze
  • possible that robot will never solve the maze, or solve it very inefficiently
  • it has only experience 0 reward, thinks that is the best it can do, no incentive to not act randomly
  • better solution is -1 reward at every time step
  • now it has incentive to solve the maze as quickly as possible
  • “Negative” and “Positive” don’t have connotations(含义) when it comes to RL agents
  • just a number on a scale
  • E.g -3 is better reward than -300
  • been debated whether or not we should override the default rewards
  • in real world environment, we would be the ones defining rewards
• State-transition probabilities
  • At first glance, might seem unnecessary
  • p(s’,r I s,a)
  • if we do action a while in state s, won’t i always go to s?
  • GridWorld: action “up”, should always take us to the square above
  • Not all environments are deterministic - have a source of randomness
  • reading of the state can be imperfect
  • may only reflect partial knowledge
• Makrov propertys
  • p[s(t+1),r(t+1)I s(t), a(t), s(t-1), a(t-1),…,(s1),a(1)]=p[s(t+1),r(t+1) I s(t), a(t)]
  • as usual by “Markov” we mean first-order Markov
  • A.K.a Markov assumption
  • p(s’,r I s,a)
  • Note: ‘ don’t mean t+1
• Discount factor
  • we would rather get $100 now than $100 years from now
  • we would like to maximize total future reward
  • the further we look into the future, the harder it is to predict
  • therefore, discount future rewards to make them “matter less”
  • we can then define the return:
• Value Function
  • Because rewards are probabilistic, and return are sums of rewards, they are also probabilistic
  • can define the expected value - call this the Value Funtion
  • The expected return from being in state s
• State-Value and Action-Value
  • State-Value: V(s), Action-Value: Q(s,a)
  • Typically use Q for learning the optimal policy (control problem)
  • Finding V or Q given a fixed policy is called the prediction problem
  • Policy denoted by π, can be deterministic or probabilistic
• When to use Q(s,a)
  • Q(s,a) is better for the control problem since it is indexed by action, tells us what the best action is given a state
  • With V(s) we need to actually do the action, and see what the next state s’ is, to determine the best action
  • Not feasible to “reset” a state in the real world
• Episodes
  • E.g one run of tic-tac-toe
  • Typically an agent will need to play many episodes to learn an optimal policy
  • Tasks that end are called episodic tasks
  • Tasks that last forever are called continuous task
  • Terminal state - the state at which an episode ends
  • Since the value function is the expected future reward after arriving in a state, the value of any terminal state is 0

2. Dynamic Programming

• Pioneered by Richard Bellman
• The bellman Equations for MDPs allow us to define the value function recursively
• if we look carefully, this is actually a linear system of equations:
  • V(s1)= $a_11$V(s1) + $a_12$V(s1) + … + $a_1N$V(sN)
  • V(s2)= $a_21$V(s2) + $a_22$V(s1) + … + $a_2N$V(sN)
• but since this is machine learning we are more interested in iterative solutions
• prediction problem: Given a polic, find the value function
• simply keep recalculating the right side and assign it to the left until there is no more change
• bellman’s equation says they should be equal
• one “trick” : don’t update set of V(s)’s only from previous set of V(s)’s, just update V(s) in-place -it’s faster
• Policy Iteration
  • For solving the control problem

    while not converged:
    Step 1) Policy evaluation on current policy
    Step 2) Policy Improvement (take the argmax over Q(s,a))
    

  • Policy iteration is inefficient
  • outer loop is iterative
  • policy evaluation(inner loop) is also iterative
  • iterative algorithm inside another
• Value Iteration
  • Instead of waiting for policy evaluation to converge just do iteration
  • overall it will still converge
  • Furthermore, we don’t need to explicitly do the policy improvement step at all
  • Since it’s the argmax, then taking the max in the policy evaluation step in equivalent
  • Taking the max appears again in Q-Learning
• lays the ground work, but is not very practical
• we need to loop through all state on every iteration
• state space may be very large or infinite
• requires us to know p(s’,r I s,a)
• calculating that can be infeasible
• doesn’t learn from experience
• MC and TD learning do, no model of the environment needed

3. Monte Carlo Methods(MC)

• Unlike DP, MC is all about learning from experience
• Expected values can be approximated by sample means
• Play a bund of episodes, gather returns, average them
• only gives us values for states we encountered
• if we never encountered a state, its value is unknown
• what about the control problem?
• we return to the ide of policy iteration
• Montel Carlo control
  1. initialize random policy
  2. while not converged:
     • play an episode, calculate returns for each state
     • Do policy improvement based on current Q(s,a)(take the argmax)
     • Look-ahead search with V(s) is not practical if we don’t have full control over environment, so we use Q(s,a)
     • Unusual/interesting: the averaged returns are for difference policies, yet it still converged
• Problem
  • MC control as given won’t always work
  • requires many episodes
  • what if we’re not doing an episodic task?
  • what if the current policy is to bump into a wall or walk around in a circle? the episode will never end
  • one solution: end the episode if we catch ourselves in a loop, give a large negative reward for that action
  • Agent won’t do it again, since policy is chosen as argmax
• Another Problem
  • MC can leave may states unexplored
  • we had never know if going to these states is better/worse than our current policy, because we have no data on them
  • one solution: the “exploring stats” method-start from a random state each episode
  • requires full control over environment, not always feasible
  • Another solution: epsilon-greedy
  • by moving randomly with small probability epsilon, we’ll eventually explore all states sufficiently

4. Temporal difference Learning(TD)

• Unique to RL
• MC: sample returns based on an episode
• TD: estimate return based on current value function estimate
• We looked at TD(0)
• instead of using G, we use r+ɣV(s’)
• Calculating means
  • basic naive way:
  • collect all samples in an array, sum them, divide by N
  • Major disadvantage - it requires us to store all samples in memory - space inefficient
  • can calculate current sample mean from last sample mean
  • Not only is it space efficient, it also looks curiously like gradient descent(in fact, it is ) we can Generalized it:
• Back to TD(0)
  • remember, instead of using G, we use r and V
  • Allows us to do true online learning
  • only need to wait until t+1 to update value for state at time t
  • MC - need to wait until entire epsiode is over
  • can learn during the episode, can be used for long or non-episodic tasks
• TD control
  • we learned 2 algorithms
  • Sarsa

Pseudocode

Q(s,a) = arbitrary, Q(terminal,a) = 0
for t=1..N
  s = start_state, a = epsilon_greedy_from(Q(s))
  while not game over:
    s', r = do_action(a)
    a' = episode_greedy_from(Q(s'))
    Q(s,a) = Q(s,a) + alpha + [r + gamma*Q(s',a')-Q(s,a)]
    s = s', a = q'

• Q-Learning
  • similar to SARSA, but an off-policy algorithm
  • it doesn’t matter what action we take next, the target remains the same since it’s a max

5. Approximation Methods

• we studied DP, MC, and TD using tabular methods
• we stored V(s) and Q(s,a) as dictionaries
• this work ok but only for small problems
• it won’t work for billion or trillion of states, and certainly not an infinite number of states
• we know that supervised learning can be used for funtion approximation
• we want to estimate V or Q
• Since reward is a real number, so is , therefore we do regression, and the appropriate loss is squared error
• Linear Models
  • we need a differential model so we can do gradient descent
  • theta is the model parameter, so that’s what we need to update
• General Models
  • In general, the model need not be linear, just differentiable like a neural network
  • All deep learning techinque can be applied batch gradient descent, momentum, dropout and other regulation, etc.
  • in general form:

    Next Steps

• Continuous state-spaces
  • light intensity in 3-D space
  • 3-D space is Continuous, light intensity is Continuous
• Continuous action-space
  • Amount of force applied to a motor
• parameterized V
• parameterized π
  • it called “policy gradient” method
• learned from our current experience only
• as we play, we accumulate(input, target) pairs
  • E.g accumulate training data
• save these in a file and do more training
• in deep learning, we loop through the same training data multiple times (called ‘epochs’)
• No reason we can’t learn from our previous experience as well
• Replay a previous episode, update params based on old episodes
• CNNs
• RNNs

• Regression, Classification

  Reference:

  Artificial Intelligence Reinforcement Learning

  Advance AI : Deep-Reinforcement Learning

  Cutting-Edge Deep-Reinforcement Learning

1. Create Msg

• folder inisde project name “msg”
• create file “Twoints.msg”
• put it the data format

TwoInts.msg

int16 a
int16 b

• we can use publish or script from the Twoints.msg(int16 a/int16 b)
• and go to CMakeList.txt, add it msg file in the “add_message_files “

CMakeList.txt

# Generate messages in the 'msg' folder
add_message_files(
  FILES
  TwoInts.msg
)

• and roscd /cd.. catkin_make

2. use it (1)

two_int_talker.py

#!/usr/bin/env python  
import random

import rospy
from std_msgs.msg import Int16
from project1_solution.msg import TwoInts

def talker():
    pub = rospy.Publisher('two_ints', TwoInts, queue_size=10)
    rospy.init_node('two_int_talker', anonymous=True)
    rate = rospy.Rate(0.5)  
    random.seed()

    while not rospy.is_shutdown():

        msg = TwoInts()

        msg.a = random.randint(1,20)
        msg.b = random.randint(1,20)
        pub.publish(msg)
        rate.sleep()


if __name__ == '__main__':
    try:
        talker()
    except rospy.ROSInterruptException:
        pass

2. use it (1)

sum.py

#!/usr/bin/env python  
import rospy

from std_msgs.msg import Int16
from project1_solution.msg import TwoInts

def cb(msg):
    global SumMsg,addPub
    SumMsg = msg.a + msg.b
    addPub.publish(SumMsg) # publish in the addpub topic("/sum")

rospy.init_node('solution')

SumMsg = Int16()
addPub = rospy.Publisher('/sum',Int16,queue_size=1)
Sub = rospy.Subscriber('/two_ints',TwoInts,cb) #cb is callback the data read it in the function

rospy.spin()

Sensor_info

SensorInformation.msg

sensor_msgs/Range sensor_data
string maker_name
uint32 part_number

sensor_info_publisher.py

#!/usr/bin/env python
import rospy
from hrwros_msgs.msg import SensorInformation
from hrwros_utilities.sim_sensor_data import distSensorData as getSensorData

def sensorInfoPublisher():
    si_publisher = rospy.Publisher('sensor_info', SensorInformation, queue_size = 10)
    rospy.init_node('sensor_info_publisher', anonymous=False)
    rate = rospy.Rate(10)

    # Create a new SensorInformation object and fill in its contents.
    sensor_info = SensorInformation()

    # Fill in the header information.
    sensor_info.sensor_data.header.stamp = rospy.Time.now()
    sensor_info.sensor_data.header.frame_id = 'distance_sensor_frame'

    # Fill in the sensor data information.
    sensor_info.sensor_data.radiation_type = sensor_info.sensor_data.ULTRASOUND
    sensor_info.sensor_data.field_of_view = 0.5 # Field of view of the sensor in rad.
    sensor_info.sensor_data.min_range = 0.02 # Minimum distance range of the sensor in m.
    sensor_info.sensor_data.max_range = 2.0 # Maximum distance range of the sensor in m.

    # Fill in the manufacturer name and part number.
    sensor_info.maker_name = 'The Ultrasound Company'
    sensor_info.part_number = 123456

    while not rospy.is_shutdown():
        # Read the sensor data from a simulated sensor.
        sensor_info.sensor_data.range = getSensorData(sensor_info.sensor_data.radiation_type,
            sensor_info.sensor_data.min_range, sensor_info.sensor_data.max_range)

        # Publish the sensor information on the /sensor_info topic.
        si_publisher.publish(sensor_info)
        # Print log message if all went well.
        rospy.loginfo("All went well. Publishing topic ")
        rate.sleep()

if __name__ == '__main__':
    try:
        sensorInfoPublisher()
    except rospy.ROSInterruptException:
        pass

sensor_info_subscriber.py

#!/usr/bin/env python
import rospy
from hrwros_msgs.msg import SensorInformation

def sensorInfoCallback(data):
    rospy.loginfo('Distance reading from the sensor is: %f', data.sensor_data.range)

def sensorInfoListener():

    # In ROS, nodes are uniquely named. If two nodes with the same
    # name are launched, the previous one is kicked off. The
    # anonymous=True flag means that rospy will choose a unique
    # name for our 'sensorInfoListener' node so that multiple listeners can
    # run simultaneously.
    rospy.init_node('sensor_info_subscriber', anonymous=False)

    rospy.Subscriber('sensor_info', SensorInformation, sensorInfoCallback)

    # spin() simply keeps python from exiting until this node is stopped
    rospy.spin()

if __name__ == '__main__':
    sensorInfoListener()

Template

template_publisher.py

#!/usr/bin/env python
import rospy
from std_msgs.msg import String

def simplePublisher():
    simple_publisher = rospy.Publisher('topic_1', String, queue_size = 10)
    rospy.init_node('node_1', anonymous=False)
    rate = rospy.Rate(10)

    topic1_content = "Welcome to Hello (Real) World with ROS!!!"

    while not rospy.is_shutdown():
        simple_publisher.publish(topic1_content)
        rate.sleep()

if __name__ == '__main__':
    try:
        simplePublisher()
    except rospy.ROSInterruptException:
        pass

template_subscriber.py

#!/usr/bin/env python
import rospy
from std_msgs.msg import String

def stringListenerCallback(data):
    rospy.loginfo('%s', data.data)

def stringListener():

    # In ROS, nodes are uniquely named. If two nodes with the same
    # name are launched, the previous one is kicked off. The
    # anonymous=True flag means that rospy will choose a unique
    # name for our 'stringListener' node so that multiple listeners can
    # run simultaneously.
    rospy.init_node('node_2', anonymous=False)

    rospy.Subscriber('topic_1', String, stringListenerCallback)

    # spin() simply keeps python from exiting until this node is stopped
    rospy.spin()

if __name__ == '__main__':
    stringListener()

Metres to feet

ConvertMetresToFeet.srv

float64 distance_metres		# Request message: Distance in (m) to be converted to feet.
---
float64 distance_feet		# Response message: Distance in feet after conversion.
bool success				# Response message: Success or failure of conversion.

metres_to_feet_server.py

#!/usr/bin/env python
# This code has been adapted from the ROS Wiki ROS Service tutorials to the context
# of this course.
# (http://wiki.ros.org/ROS/Tutorials/WritingServiceClient%28python%29)

from hrwros_msgs.srv import ConvertMetresToFeet, ConvertMetresToFeetRequest, ConvertMetresToFeetResponse
import rospy
import numpy as np

_CONVERSION_FACTOR_METRES_TO_FEET = 3.28 # Metres -> Feet conversion factor.

# Service callback function.
def process_service_request(req):

    # Instantiate the response message object.
    res = ConvertMetresToFeetResponse()

    # Perform sanity check. Allow only positive real numbers.
    # Compose the response message accordingly.
    if(req.distance_metres < 0):
        res.success = False
        res.distance_feet = -np.Inf # Default error value.
    else:
        res.distance_feet = _CONVERSION_FACTOR_METRES_TO_FEET * req.distance_metres
        res.success = True

    #Return the response message.
    return res

def metres_to_feet_server():
    # ROS node for the service server.
    rospy.init_node('metres_to_feet_server', anonymous = False)

    # Create a ROS service type.
    service = rospy.Service('metres_to_feet', ConvertMetresToFeet, process_service_request)

    # Log message about service availability.
    rospy.loginfo('Convert metres to feet service is now available.')
    rospy.spin()

if __name__ == "__main__":
    metres_to_feet_server()

metres_to_feet_client.py

#!/usr/bin/env python
# This code has been adapted from the ROS Wiki ROS Service tutorials to the context
# of this course.
# (http://wiki.ros.org/ROS/Tutorials/WritingServiceClient%28python%29)

import sys
import rospy
from hrwros_msgs.srv import ConvertMetresToFeet, ConvertMetresToFeetRequest, ConvertMetresToFeetResponse

def metres_to_feet_client(x):
    # First wait for the service to become available.
    rospy.loginfo("Waiting for service...")
    rospy.wait_for_service('metres_to_feet')
    try:
        # Create a service proxy.
        metres_to_feet = rospy.ServiceProxy('metres_to_feet', ConvertMetresToFeet)

        # Call the service here.
        service_response = metres_to_feet(x)

        print("I only got here AFTER the service call was completed!")

        # Return the response to the calling function.
        return service_response

    except rospy.ServiceException, e:
        print "Service call failed: %s"%e

if __name__ == "__main__":

    # Initialize the client ROS node.
    rospy.init_node("metres_to_feet_client", anonymous = False)

    # The distance to be converted to feet.
    dist_metres = 0.25

    rospy.loginfo("Requesting conversion of %4.2f m to feet"%(dist_metres))

    # Call the service client function.
    service_response = metres_to_feet_client(dist_metres)

    # Process the service response and display log messages accordingly.
    if(not service_response.success):
        rospy.logerr("Conversion unsuccessful! Requested distance in metres should be a positive real number.")
    else:
        rospy.loginfo("%4.2f(m) = %4.2f feet"%(dist_metres, service_response.distance_feet))
        rospy.loginfo("Conversion successful!")

counter_with_delay

launch file

<?xml version="1.0"?>
<launch>
  <!-- Argument to the launch file.-->
  <arg name="counter_delay_parameter" default="1.0"/>

  <!-- Start the metres_to_feet service server ROS node.-->
  <node name="metres_to_feet" pkg="hrwros_week1" type="metres_to_feet_server.py"
    output="screen"/>

  <!-- Start the action server ROS node if the start_as argument is true /-->
  <node name="counter_with_delay" pkg="hrwros_week1" type="counter_with_delay_as.py"
    output="screen">
    <param name="counter_delay" type="double" value="$(arg counter_delay_parameter)"/>
  </node>

</launch>

CounterWithDelay.action

uint32 num_counts			# Goal message: number of counts to count up to.
---
string result_message		# Result message: simple string message for the result.
---
uint32 counts_elapsed		# Feedback message: number of counts elapsed.

counter_with_delay_as.py

#! /usr/bin/env python

# This code has been adapted from the ROS Wiki actionlib tutorials to the context
# of this course.
# (http://wiki.ros.org/hrwros_msgs/Tutorials)

import rospy

import actionlib

from hrwros_msgs.msg import CounterWithDelayAction, CounterWithDelayFeedback, CounterWithDelayResult

class CounterWithDelayActionClass(object):
    # create messages that are used to publish feedback/result
    _feedback = CounterWithDelayFeedback()
    _result = CounterWithDelayResult()

    def __init__(self, name):
        # This will be the name of the action server which clients can use to connect to.
        self._action_name = name

        # Create a simple action server of the newly defined action type and
        # specify the execute callback where the goal will be processed.
        self._as = actionlib.SimpleActionServer(self._action_name, CounterWithDelayAction, execute_cb=self.execute_cb, auto_start = False)

        # Start the action server.
        self._as.start()
        rospy.loginfo("Action server started...")

    def execute_cb(self, goal):
        # OPTIONAL: NOT GRADED Check if the parameter for the counter delay is available on the parameter server.
        # if not rospy.has_param("<write your code here>"):
        #     rospy.loginfo("Parameter %s not found on the parameter server. Using default value of 1.0s for counter delay.","counter_delay")
        #     counter_delay = 1.0
        # else:
        # Get the parameter for delay between counts.
        counter_delay_value = rospy.get_param("<write your code here>")
        rospy.loginfo("Parameter %s was found on the parameter server. Using %fs for counter delay."%("counter_delay", counter_delay_value))

        # Variable to decide the final state of the action server.
        success = True

        # publish info to the console for the user
        rospy.loginfo('%s action server is counting up to  %i with %fs delay between each count' % (self._action_name, goal.num_counts, counter_delay_value))

        # start executing the action
        for counter_idx in range(0, goal.num_counts):
            # check that preempt has not been requested by the client
            if self._as.is_preempt_requested():
                rospy.loginfo('%s: Preempted' % self._action_name)
                self._as.set_preempted() # Preempted 획득하다
                success = False
                break
            # publish the feedback
            self._feedback.counts_elapsed = counter_idx
            self._as.publish_feedback(self._feedback)
            # Wait for counter_delay s before incrementing the counter.
            <write your code here>

        if success:
            self._result.result_message = "Successfully completed counting."
            rospy.loginfo('%s: Succeeded' % self._action_name)
            self._as.set_succeeded(self._result)

if __name__ == '__main__':
    # Initialize a ROS node for this action server.
    rospy.init_node('counter_with_delay')

    # Create an instance of the action server here.
    server = CounterWithDelayActionClass(rospy.get_name())
    rospy.spin()

counter_with_delay_ac.py

#! /usr/bin/env python
from __future__ import print_function
import rospy

# Brings in the SimpleActionClient
import actionlib

# Brings in the messages used by the CounterWithDelay action, including the
# goal message and the result message.
from hrwros_msgs.msg import CounterWithDelayAction, CounterWithDelayGoal, CounterWithDelayResult

def counter_with_delay_client():
    # Creates the SimpleActionClient, passing the type of the action
    # (CounterWithDelayAction) to the constructor.
    client = actionlib.SimpleActionClient('counter_with_delay', CounterWithDelayAction)

    # Waits until the action server has started up and started
    # listening for goals.
    rospy.loginfo("Waiting for action server to come up...")
    client.wait_for_server()

    num_counts = 10

    # Creates a goal to send to the action server.
    goal = CounterWithDelayGoal(num_counts)

    # Sends the goal to the action server.
    client.send_goal(goal)

    rospy.loginfo("Goal has been sent to the action server.")

    # Waits for the server to finish performing the action.
    client.wait_for_result()

    # Prints out the result of executing the action
    return client.get_result()  # A CounterWithDelayResult

if __name__ == '__main__':
    try:
        # Initializes a rospy node so that the SimpleActionClient can
        # publish and subscribe over ROS.
        rospy.init_node('counter_with_delay_ac')
        result = counter_with_delay_client()
        rospy.loginfo(result.result_message)
    except rospy.ROSInterruptException:
        print("program interrupted before completion", file=sys.stderr)

Fundamental Concepts

• ROS file system -nomenclature
  • ROS workspace -catkin workspace
    • src space, devel space, build space, install space
  • ROS packages
    • organize different functional modules
• ROS workspace is a folder to organize ROS project files
• catkin - a build tool that compiles source files to binaries
• catkin_workspace - ROS workspace where catkin is used as the build tool

• install dependencies of a ROS package(listed in package.xml)

cd <path_to_folder_with_ROS_packages(s)>
rosdep install <package_name>

• install dependencies of all packages in our source space

cd <path_to_ros_workspace/src>
rosdep install --from-paths . --ignore-src -y

• we dont need the source files of all dependencies of our ROS package

• Devel space

  • Binary executable files for tesing our development
  • files for setting up project specific ROS environment
  • source ROS environment for project in catkin workspace

source <path_to_workspace/devel/setup.bash

ROS launch

• Group Multiple Ros nodes in one file
  • launch file in xml format
• start up all nodes in the launch file
  • arguments, node specific paramters, namespaces
• also possible to include launch files from other packages in the same launch file

example

<launch>
  <node name="counter_with_delay" pkg="hrwros" type="counter_with_delay_as.py" output="screen">
    <param name="counter_delay" type="double" vale="$(arg counter_delay_parameter)"/>

ROS Actions

• No waiting until an execution is complete(non-blocking)
  • waiting is an option if required
• A generalized request-response system - the client-sever infrastructure in ROS
• Actions are defined by the three message types: Goal(request), Result(response) and Feedback
• Requirement: counter with a 1s delay between each count
  • Goal Message - Number of count up to (uint32)
  • Result message - state message(string)
  • feedback message - number of counts completed(unit32)

CounterWithDelay.action

unit32 num_counts <- Goal Field
---
string result_message <- result field
---
unit32 counts_elapsed <- feedback field

• Ros action definitions reside in the ROS package with project specific message definitions
  • /action folder
• generate action messages manually
  • rosrun actionlib_msgs genaction.py
• show the contents of an action message
  • rosmsg show _ msgs/

Processing a goal request

• goalCallbcak function processes a goal request
  • a goal can be pre-empted(and cancelled) before completion
• goal statuses: ACTIVE, SUCCEEDED, ABORTED(실패)

client = actionlb.SimpleActionClient(counter_with_delay,CounterWithDelayAction)

client.wait_for_server()

goal = CounterWithDelayGoal(num_counts=10)
client.send_goal(goal)

ROS Service

• Event-Based execution for ROS applications

float64 measurement_metres <- request field
---
float64 measurement_feet <- response field
Bool success