for Robot Artificial Inteligence

6. path planning

06 Oct 2019 | ROBOTICS

path planning

  • Geometrical description of the robot’s movement in space
  • smooth transition between two or more target points
  • A unique geometrical path can be described by different time dependent trajectories
  • Nomenclature(命名法)
    • PTP
    • Line
    • Circle
    • Splines

1. Point to Point(PTP)

  • Linear interpolation of joint axes
  • t is just parameter describing the curve from zero to one only geometrically
  • no direct control over TCP position and orientation
  • they have no control over the actual path of the end effector of the robot
  • cyclic call of the kinematic
  • time-optimal movement(move as fast as joint axes can)
  • only movement that can modify joint configuration

2. path-interpolated movements

  • orientation cannot be interpolated linearly when using Euler angle

3. Quaternions

  • convenient way to express an orientation
  • commutative means that (교환법칙이 성립할 때를 의미함)
  • Vs Euler angles
    • unique representation of rotation
    • can be easily interpolated
  • Vs Rotation Matrix
    • more compact representation
    • easy to interpolate
    • do not suffer from numerical approximation
  • slerp(Spherical Linear interpolation:구면보간법)
    • Optimal(shortest) path between $q_1$ and $q_2$

4. Line and Circle

  • Interpolate position linearly

  • Circle detail
    • point must be non collinear(동일선상)
    • N = $v_1$ x $v_2$
  • direction N stays the same, only magnitude changed
  • normalize N after the product

5. spline(매끄러운 곡선)

  • The operator needs a “smooth” path between points

5.1 Practice

  • place control point along targets

6 transition(변화)

  • round edge
  • sharp edges are a problem
    • smooth geometrical path
    • increase path speed
    • reduces mechanical wear

7 path length and corrections

  • analytical calculation for lines and circles

  • numerically calculation for PTP and splines

  • external path corrections

  • happen at runtime, not planned, not predictable
  • typical case : conveyor tracking(increased productivity)
  • position limits must be monitored at runtime
  • path length changes, so dynamic limits must also be monitored at runtime

Comment  Read more

6. Optimization concave and convex

06 Oct 2019 | Optimization method

Graph

Convex and concave

Comment  Read more

6. object_generator

06 Oct 2019 | ROS

Object Generator

Object Generator

```python #!/usr/bin/env python

import math import numpy

import geometry_msgs.msg import moveit_commander import rospy import tf

from visualization_msgs.msg import Marker from visualization_msgs.msg import MarkerArray

def convert_to_message(T): t = geometry_msgs.msg.Pose() position = tf.transformations.translation_from_matrix(T) orientation = tf.transformations.quaternion_from_matrix(T) t.position.x = position[0] t.position.y = position[1] t.position.z = position[2] t.orientation.x = orientation[0] t.orientation.y = orientation[1] t.orientation.z = orientation[2] t.orientation.w = orientation[3]
return t

class ObstacleGenerator(object):

def __init__(self):
    self.scene = moveit_commander.PlanningSceneInterface()
    self.marker_pub = rospy.Publisher("/visualization_marker_array", MarkerArray, queue_size=1)

def remove_planning_obs(self):
    self.scene.remove_world_object("obs1")
    self.scene.remove_world_object("obs2")
    self.scene.remove_world_object("obs3")
    self.scene.remove_world_object("obs4")

def no_obs(self):
    self.scene.remove_world_object("obs1")
    self.scene.remove_world_object("obs2")
    self.scene.remove_world_object("obs3")
    self.scene.remove_world_object("obs4")

    self.scene.remove_world_object("box1")
    self.scene.remove_world_object("box2")
    self.scene.remove_world_object("box3")
    self.scene.remove_world_object("box4")

    obs = MarkerArray()
    obj1 = Marker()
    obj1.ns = 'obstacle'
    obj1.id = 1
    obj1.action = 2;
    obs.markers.append(obj1)
    obj2 = Marker()
    obj2.ns = 'obstacle'
    obj2.id = 2
    obj2.action = 2;
    obs.markers.append(obj2)
    obj3 = Marker()
    obj3.ns = 'obstacle'
    obj3.id = 3
    obj3.action = 2;
    obs.markers.append(obj3)
    obj4 = Marker()
    obj4.ns = 'obstacle'
    obj4.id = 4
    obj4.action = 2;
    obs.markers.append(obj4)
    self.marker_pub.publish(obs)

def simple_obs(self):
    self.no_obs()
    pose_stamped = geometry_msgs.msg.PoseStamped()
    pose_stamped.header.frame_id = "world_link"
    pose_stamped.header.stamp = rospy.Time(0)

    pose_stamped.pose = convert_to_message( tf.transformations.translation_matrix((0.5, 0.25, 0.2)) )
    self.scene.add_box("obs1", pose_stamped,(0.1,0.1,1))
    self.scene.add_box("box1", pose_stamped,(0.05, 0.05, 0.95))

    obs = MarkerArray()
    obj = Marker()
    obj.header.frame_id = "world_link"
    obj.header.stamp = rospy.Time(0)
    obj.ns = 'obstacle'
    obj.id = 1
    obj.type = Marker.CUBE
    obj.action = Marker.ADD
    obj.pose = convert_to_message( tf.transformations.translation_matrix((0.5, 0.25, 0.2)) )
    obj.scale.x = 0.1
    obj.scale.y = 0.1
    obj.scale.z = 1.0
    obj.color.r = 0.0
    obj.color.g = 1.0
    obj.color.b = 0.0
    obj.color.a = 1.0
    obs.markers.append(obj)
    self.marker_pub.publish(obs)

def complex_obs(self):
    self.no_obs()
    pose_stamped = geometry_msgs.msg.PoseStamped()
    pose_stamped.header.frame_id = "world_link"
    pose_stamped.header.stamp = rospy.Time(0)
    pose_stamped.pose = convert_to_message( tf.transformations.translation_matrix((0.5, 0.25, 0.4)) )
    self.scene.add_box("obs1", pose_stamped,(0.1,0.1,0.8))
    self.scene.add_box("box1", pose_stamped,(0.05,0.05,0.75))
    pose_stamped.pose = convert_to_message( tf.transformations.translation_matrix((0.5, 0.0, 0.8)) )
    self.scene.add_box("obs2", pose_stamped,(0.1,0.5,0.1))
    self.scene.add_box("box2", pose_stamped,(0.05,0.45,0.05))

    obs = MarkerArray()
    obj1 = Marker()
    obj1.header.frame_id = "world_link"
    obj1.header.stamp = rospy.Time(0)
    obj1.ns = 'obstacle'
    obj1.id = 1
    obj1.type = Marker.CUBE
    obj1.action = Marker.ADD
    obj1.pose = convert_to_message( tf.transformations.translation_matrix((0.5, 0.25, 0.4)) )
    obj1.scale.x = 0.1
    obj1.scale.y = 0.1
    obj1.scale.z = 0.8
    obj1.color.r = 0.0
    obj1.color.g = 1.0
    obj1.color.b = 0.0
    obj1.color.a = 1.0
    obs.markers.append(obj1)

    obj2 = Marker()
    obj2.header.frame_id = "world_link"
    obj2.header.stamp = rospy.Time(0)
    obj2.ns = 'obstacle'
    obj2.id = 2
    obj2.type = Marker.CUBE
    obj2.action = Marker.ADD
    obj2.pose = convert_to_message( tf.transformations.translation_matrix((0.5, 0.0, 0.8)) )
    obj2.scale.x = 0.1
    obj2.scale.y = 0.5
    obj2.scale.z = 0.1
    obj2.color.r = 0.0
    obj2.color.g = 1.0
    obj2.color.b = 0.0
    obj2.color.a = 1.0
    obs.markers.append(obj2)

    self.marker_pub.publish(obs)

def super_obs(self):
    self.no_obs()
    pose_stamped = geometry_msgs.msg.PoseStamped()
    pose_stamped.header.frame_id = "world_link"
    pose_stamped.header.stamp = rospy.Time(0)
    pose_stamped.pose = convert_to_message( tf.transformations.translation_matrix((0.5, 0.25, 0.4)) )
    self.scene.add_box("obs1", pose_stamped,(0.1,0.1,0.8))
    self.scene.add_box("box1", pose_stamped,(0.05,0.05,0.75))
    pose_stamped.pose = convert_to_message( tf.transformations.translation_matrix((0.5, 0.0, 0.8)) )
    self.scene.add_box("obs2", pose_stamped,(0.1,0.5,0.1))
    self.scene.add_box("box2", pose_stamped,(0.05,0.45,0.05))
    pose_stamped.pose = convert_to_message( tf.transformations.translation_matrix((0.5, -0.25, 0.4)) )
    self.scene.add_box("obs3", pose_stamped,(0.1,0.1,0.8))
    self.scene.add_box("box3", pose_stamped,(0.05,0.05,0.75))
    pose_stamped.pose = convert_to_message( tf.transformations.translation_matrix((0.5, 0.0, 0.3)) )
    self.scene.add_box("obs4", pose_stamped,(0.1,0.5,0.1))
    self.scene.add_box("box4", pose_stamped,(0.05,0.45,0.05))

    obs = MarkerArray()
    obj1 = Marker()
    obj1.header.frame_id = "world_link"
    obj1.header.stamp = rospy.Time(0)
    obj1.ns = 'obstacle'
    obj1.id = 1
    obj1.type = Marker.CUBE
    obj1.action = Marker.ADD
    obj1.pose = convert_to_message( tf.transformations.translation_matrix((0.5, 0.25, 0.4)) )
    obj1.scale.x = 0.1
    obj1.scale.y = 0.1
    obj1.scale.z = 0.8
    obj1.color.r = 0.0
    obj1.color.g = 1.0
    obj1.color.b = 0.0
    obj1.color.a = 1.0
    obs.markers.append(obj1)

    obj2 = Marker()
    obj2.header.frame_id = "world_link"
    obj2.header.stamp = rospy.Time(0)
    obj2.ns = 'obstacle'
    obj2.id = 2
    obj2.type = Marker.CUBE
    obj2.action = Marker.ADD
    obj2.pose = convert_to_message( tf.transformations.translation_matrix((0.5, 0.0, 0.8)) )
    obj2.scale.x = 0.1
    obj2.scale.y = 0.5
    obj2.scale.z = 0.1
    obj2.color.r = 0.0
    obj2.color.g = 1.0
    obj2.color.b = 0.0
    obj2.color.a = 1.0
    obs.markers.append(obj2)

    obj3 = Marker()
    obj3.header.frame_id = "world_link"
    obj3.header.stamp = rospy.Time(0)
    obj3.ns = 'obstacle'
    obj3.id = 3
    obj3.type = Marker.CUBE
    obj3.action = Marker.ADD
    obj3.pose = convert_to_message( tf.transformations.translation_matrix((0.5, -0.25, 0.4)) )
    obj3.scale.x = 0.1
    obj3.scale.y = 0.1
    obj3.scale.z = 0.8
    obj3.color.r = 0.0
    obj3.color.g = 1.0
    obj3.color.b = 0.0
    obj3.color.a = 1.0
    obs.markers.append(obj3)

    obj4 = Marker()
    obj4.header.frame_id = "world_link"
    obj4.header.stamp = rospy.Time(0)
    obj4.ns = 'obstacle'
    obj4.id = 4
    obj4.type = Marker.CUBE
    obj4.action = Marker.ADD
    obj4.pose = convert_to_message( tf.transformations.translation_matrix((0.5, 0.0, 0.3)) )
    obj4.scale.x = 0.1
    obj4.scale.y = 0.5
    obj4.scale.z = 0.1
    obj4.color.r = 0.0
    obj4.color.g = 1.0
    obj4.color.b = 0.0
    obj4.color.a = 1.0
    obs.markers.append(obj4)

    self.marker_pub.publish(obs)

Comment  Read more

22. Plug in Neural network and racap OPENAI

06 Oct 2019 | Reinforcement Learning

Plugging in a neural network

  • what might happen if we plugged in a neural network instead of a linear model?
  • we have seen how we can plug in a tensorflow model into q_learning.py script(Tensorflow_warmup)
  • put them together
  • try removing RBF layer since presumably(很可能) a deep neural network will automatically learn features

catastrophic(灾难性的) Forgetting

  • lots of attention recently, in relation to “transfer learning”
  • train AI on one game, keep some weights, train on another game
  • showed the neural net can be trained such that the AI still performs well on the first game
  • so it didn’t just learn the 2nd game and forget the 1st game
  • this is noteworthy(值得注意的) because it’s not how neural networks work by default
  • more typically we would expect the neural net to forget how to play the 1st game
  • Doesn’t only apply across different tasks
  • Stochastic/ batch gradient descent -> cost can “jump” around
  • seems more pronounced on highly nonlinear regression problems
  • we’d like the data in cost function to represent true distribution of data
  • even using all training data simultaneously is only an approximation of the “true” data
  • E.g. clinical trial for a drug, we sample 1000 subjects. we hope they are representative of the population as a whole
  • So when we use batch/Stochastic GD, the approximation becomes even worse
  • in RL, it’s even worse because there’s a large bias in the ordering of training samples
  • we always proceed from start state to end state
  • No Randomization, which is recommended in SGD/BGD
  • Also, not a true gradient because the target uses the model to make a prediction

Try dropout

  • Dropout has been shown to help
  • some weights will not be updated for any particular training iteration
  • try other types of regularization and architectures

Open AI gym & RBF Network summary

  • Building Gradually more and more complex RL agents, using what we already know

OpenAI Gym

  • how to load an environment
  • inspect its state space, action space
  • play an episode
  • save a video

CartPole

  • Random search
  • Easy to conceptualize, no complicated math
  • choose random weights until we find something good

Tabular Q-Learning

  • Q-Learning with binned states
  • Allows us to use tabular method
  • Re-acquaint(再使熟悉) us with q_learning/TD(0)

RBF Networks

  • historically interesting, related to both SVMs and neural networks
  • Could use nonlinear features, but still have a linear gradient descent algorithm
  • Other Perspective: we’re still doing linear regression, just with a different feature expansion

Mountain Car

  • Opportunity to test Q-Learning with RBF networks
  • First use of approximation methods
  • Choosing RBF exemplars, combining features, other sklearn tools

Back to CartPole

  • then applied the same algorithm to CartPole
  • state space extends to infinity
  • OpanAI gym samples from state space uniformly, not wrt(with regard to(…에 관해서)) probability of actually being in a state
  • Not good exemplar
  • Also: built our own linear function approximator using numpy

``` Reference:

Artificial Intelligence Reinforcement Learning

Advance AI : Deep-Reinforcement Learning

Cutting-Edge Deep-Reinforcement Learning

Comment  Read more

6 Optimization Condition

06 Oct 2019 | Optimization method

Optimization Condition

First Order Necessary Condition(FONC)

Geometry Illustration

FONC for interior case

Example

Second order necessary condition(SONC)

SONC for interior case

The necessary conditions are not sufficient

Second-order sufficient condition (SOSC)

Example

Remarks on roles of optimality(最优性) conditions

Reference

Optimization method - Standford University

https://www.youtube.com/results?search_query=convex+function

Comment  Read more