Manifest: Installation instructions for Unsupervised.ai
======================

The repository contains all the info to install and setup development/production environment at unsupervised.ai.

## Table of content

- [Installation](#installation)
    - [For Developers](#for-developers)
        - [Initial Developer Install](#initial-install-for-developers)
        - [SSH Setup](#ssh-setup-for-developers)
        - [Git configuration](#git-configuration-for-developers)
        - [Installing ROS](#installing-ros-for-developers)
            - [Installing 'ros-kinetic-desktop-full'](#installing-ros-kinetic-desktop-full-for-developers)
            - [Installing ROS related packages](#installing-ros-related-packages-for-developers)
        - [Install and setup your Catkin Workspace](#install-and-setup-your-catkin-workspace-for-developers)
        - [Install and configure python-uai-utils](#install-and-configure-python-uai-utils-for-developers)
        - [Install the gazebo models](#install-the-gazebo-models)
        - [Installing NVM, NPM and NodeJS](#installing-nvm-npm-nodejs-for-developers)
        - [Add user to USB perm group](#add-user-to-usb-perm-group)
        - [Compile ROS Packages](#compile-the-ros-packages)
    - [JETSON](#for-jetsons)
        - [From a copy](#from-a-copy)
            - [Get into recovery mode](#get-into-recovery-mode)
            - [What's installed already](#whats-installed-on-the-jetson-when-you-clone-it-from-the-mother-board)
            - [Change the uai config file](#change-the-uai-config-file)
            - [Check if everything works](#check-if-everything-works)
        - [From scratch](#from-scratch)
- [Launch a simulation](#launch-a-simulation)
- [Launch RVIZ](#use-rviz)
- [Unit Testing and Integration testing](#unit-testing-and-integration-testing)
- [Useful git tools](#useful-git-tools)
- [Miscellaneous](#miscellaneous)

## Installation

This will guide you through the process of installing `ROS`, and all the tools needed to run the robots in simulation or IRL.

This tutorial has been made `on` and `for` `Ubuntu 16.04 Xenial`
### For Developers
If you're aiming to do some development for Unsupervised, You'll need many tools related to `ROS`.
#### Initial Install for developers

First, update the package list and then the packages themselves
```bash
sudo apt-get update && sudo apt-get upgrade
```

Next, install:
vim (A text editor. You can replace it with `nano` or `emacs`)
git (Version Control Software)
pip (Python's package manager)
```bash
sudo apt-get install git python-pip vim
```
#### SSH Setup for developers
Here, You will generate a new SSH key and upload it to github.

Open your terminal and enter this command (replace the email with yours)
This will generate a rsa key used by ssh to identify you.
```bash
ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
```
If you don't want to change the location of where the key is saved, just hit enter and skip the messages.
You can also skip the passphrase part if you want to.

Then, start the ssh-agent in the background
```bash
eval "$(ssh-agent -s)"
```

And give it your private key
```bash
ssh-add ~/.ssh/id_rsa
```

Finally, add your ssh key to Github.

You can cat the id_rsa.pub file and copy it manually or use `xclip`:

```bash
sudo apt-get install xclip
# Downloads and installs xclip. If you don't have `apt-get`, you might need to use another installer (like `yum`)
xclip -sel clip < ~/.ssh/id_rsa.pub
# Copies the contents of the id_rsa.pub file to your clipboard
```
Then just go to [your profile](https://github.com/settings/keys) and add a new ssh key

If you need any further help, please follow this [tutorial](https://help.github.com/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent/)

#### Git configuration for developers
You'll need next to configure git's global settings
Run these commands, Replacing with your email and name. 
```bash
  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"
```

You'll also need to configure the mode git needs to push. Just run
```bash
git config --global push.default simple
```

#### Installing ROS for developers
In this part, we will install `ROS Kinetic`, The latest version of ROS.
##### Installing ros-kinetic-desktop-full for developers

First of all, configure your computer to accept packages from `packages.ros.com`
```bash
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
```

Next, setup the keys
```bash
sudo apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-key 421C365BD9FF1F717815A3895523BAEEB01FA116
```
If you experience issues connecting to the keyserver, you can try substituting hkp://pgp.mit.edu:80 or hkp://keyserver.ubuntu.com:80 in the previous command. 

First, make sure your Debian package index is up-to-date:
```bash
sudo apt-get update
```

Then install all the `ros` packages
```bash
sudo apt-get install ros-kinetic-desktop-full
```

You then need to initialize `rosdep` (ros dependencies manager)
```bash
sudo rosdep init
rosdep update
```

Once done, we'll have the env var automagically added on CLI startup.

If you're using `bash`:
```bash
echo "source /opt/ros/kinetic/setup.bash" >> ~/.bashrc
source ~/.bashrc
```

If you're using `zsh`:
```bash
echo "source /opt/ros/kinetic/setup.zsh" >> ~/.zshrc
source ~/.zshrc
```

##### Installing ros related packages for developers

```bash
sudo apt-get install python-rosinstall python-rosinstall-generator python-wstool build-essential
```

Many of the packages needed to run the robots aren't installed when `ros` is installed. You'll need to run this command:
```bash
sudo apt-get update && sudo apt-get install \
    javascript-common \
    libjs-jquery \
    libjs-sphinxdoc \
    libjs-underscore \
    python-catkin-lint \
    python-catkin-lint \
    python-catkin-tools \
    python-concurrent.futures \
    python-osrf-pycommon \
    python-rosdep \
    python-trollius \
    python-wstool \
    python-pandas \
    ros-kinetic-gazebo-ros-control \
    ros-kinetic-gazebo-ros-control \
    ros-kinetic-gazebo-ros-pkgs \
    ros-kinetic-gazebo-ros-pkgs \
    ros-kinetic-geometry-msgs \
    ros-kinetic-joy \
    ros-kinetic-ros-control \
    ros-kinetic-ros-controllers \
    ros-kinetic-rosserial-arduino \
    ros-kinetic-serial \
    ros-kinetic-teleop-tools \
    ros-kinetic-teleop-twist-joy \
    ros-kinetic-xacro \
    ros-kinetic-web-video-server \
    ros-kinetic-teb-local-planner \
    ros-kinetic-hector-slam \
    ros-kinetic-map-server \
    ros-kinetic-amcl \
    ros-kinetic-tf2-sensor-msgs \
    ninja-build \ 
    ros-kinetic-sick-tim \ 
    ros-kinetic-rplidar-ros
```

##### Installation check for developers
Check if the installation went fine by running
```bash
source /opt/ros/kinetic/setup.bash
roscore & rosrun gazebo_ros gazebo
```
This should load up gazebo on an empty map.

#### Install and setup your Catkin Workspace for developers

First of all, make a folder for the catkin workspace
```bash
mkdir catkin_ws
cd catkin_ws
```
You can name the folder however you'd like but be sure make it match in the other commands.

Clone the manifest repository containing the required info about repositories etc
```bash
git clone git@github.com:unsupervisedai/manifest.git
```

Clone the entire workspace (all the needed repositories) with wstool
```bash
wstool init src
wstool merge -t src manifest/unsupervised.rosinstall
wstool update -t src
```

If you need to update your workspace, just run
```  bash  
wstool update -t src
```

#### Configure your unsupervised config file for developers
The uai_robotconfig contains variables needed to run the simulation. you'll need to copy it to your home first, and set the correct values to it.

Create a cache folder:
```bash
mkdir -p ~/.uai/data/
mkdir -p ~/.uai/data/maps/
```
Copy the config file to the `.uai` folder
```bash
cp manifest/configs/.uai_config ~/.uai/config
```

Open the file and update with your ROBOT_ID (Found in [this google document](https://docs.google.com/spreadsheets/d/1MaZRIkmbFmi4l9iYTp5piCtDg7zeYdqTt0d8mo5oGy4/edit?usp=sharing))
The joystick ID (Plug a xbox controler and look inside `/dev` for a `jsX`)

Add to the ~/.bashrc the .uai/config file
```bash
echo "source $HOME/.uai/config" >> ~/.bashrc
source ~/.bashrc
```
or, if you're using `zsh`:
```bash
echo "source $HOME/.uai/config" >> ~/.zshrc
source ~/.zshrc
```

#### Install and configure python-uai-utils for developers
```bash
cd $CATKIN_WS/src/python-uai-utils/
sudo python setup.py install
sudo pip install -r requirements.txt
sudo pip install boto3
```

#### Install the gazebo models
```bash
cd $CATKIN_WS/src/gazebo_models
./install.sh
cd $CATKIN_WS
```

#### Installing NVM-NPM-NodeJS for developers
```bash
cd $CATKIN_WS/src/front-end
curl -sL https://raw.githubusercontent.com/creationix/nvm/v0.31.0/install.sh -o install_nvm.sh
bash install_nvm.sh
rm install_nvm.sh
```
source your bash again
```
source ~/.bashrc
```
install node : 
```
nvm install 8.9
```
set default
```
nvm alias default 8.9
nvm use default
```
This has installed Nodejs 8.9 for bash.
If you're running zsh, please run

```bash
echo 'export NVM_DIR="/home/seluj78/.nvm" && [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"' >> ~/.zshrc && source ~/.zshrc
```
#### Add user to USB perm group
Add yourself to the dialout group to access the USB port:
```
sudo adduser $USER dialout
```
Reboot the computer:
```
sudo reboot
```

#### Compile the ROS packages
Go in your catkin workspace, update the sources and build all the packages. This may take a while
```bash
cd $CATKIN_WS
wstool update -t src
catkin build
```

Configure your .bashrc to source automatically your ROS environment
```bash
echo "source $CATKIN_WS/devel/setup.bash" >> ~/.bashrc
source ~/.bashrc
```

Configure your .zshrc to source automatically your ROS environment
```bash
echo "source $CATKIN_WS/devel/setup.zsh" >> ~/.zshrc
source ~/.zshrc
```

### For Jetsons
To setup a new nvidia jetson, there's two ways. From scratch, or from a copy of a `mother` jetson board.

#### From a copy
One of the Jetson board is labeled `MOTHER` and contains the original system image that you can copy following [this procedure](https://elinux.org/Jetson/TX2_Cloning).

##### Get into recovery mode
```
1. Power down the device. If connected, remove the AC adapter from the device. The device must be powered OFF, and not in a suspend or sleep state.
2. Connect the Micro-B plug on the USB cable to the Recovery (USB Micro-B) Port on the device and the other end to an available USB port on the host PC.
3. Connect the power adapter to the device.
4. With the system powered on:
    * Press and hold the RECOVERY FORCE button
    * While depressing the RECOVERY FORCE button, press and release the RESET button.
    • Wait 2 seconds and release the RECOVERY FORCE button.
```
##### Whats installed on the jetson when you clone it from the mother board

When freshly cloned, the new `TX2` board will have everything ready for you. The catking workspace is setup in `/home/jetson-uai/catkin_workspace` and everything inside it is built. You might want to run a quick `wstool update -t src` inside the workspace to be sure all the repositories have the latest code.

The default web browser is set to `Firefox` but the robot frontend is currently having trouble with it. 

git on the jetson is configured with an ssh key to jules@unsupervised.ai. This will be changed in the future with dev@unsupervised.ai

Two bash are installed: `zsh` and `bash`. `zsh` is the default one but can be changed if desired..

The `.zshrc` contains:

```bash

# Sourcing bash files
source /opt/ros/kinetic/setup.zsh
source "$HOME/.uai/config"
source "$HOME/catkin_workspace/devel/setup.zsh"


# Aliases
alias rm="rmtrash"
alias rmdir="rmdirtrash"
alias gww="gcc -Werror -Wall -Wextra"
alias gcom="git commit -a -m"
alias gadd="git add -A" 
alias tb="nc termbin.com 9999"

source ~/.nvm/nvm.sh

ip="$(ifconfig | grep -A 1 'wlan0' | tail -1 | cut -d ':' -f 2 | cut -d ' ' -f 1)"
export ROS_IP=$ip
export ROS_MASTER_URI="http://$ROS_IP:11311"
```

The `.bashrc` contains 

```bash
# Aliases
alias rm="rmtrash"
alias rmdir="rmdirtrash"
alias tb="nc termbin.com 9999"

# Sourcing bash files
source "$HOME/.uai/config"
source /opt/ros/kinetic/setup.bash
source "$HOME/catkin_workspace/devel/setup.bash"

source ~/.nvm/nvm.sh

ip="$(ifconfig | grep -A 1 'wlan0' | tail -1 | cut -d ':' -f 2 | cut -d ' ' -f 1)"
export ROS_IP=$ip
export ROS_MASTER_URI="http://$ROS_IP:11311"
```

`nodejs` 8.9.4 is installed on the jetson.

Termbin is also installed to quickly transfer logs

I've replaced /bin/rm with rmtrash just in case. See [doc](https://github.com/PhrozenByte/rmtrash) for more info

#### Change the uai config file

Open with your favorite editor the file `~/.uai/config` and replace `ID` with your robot ID found on [this gdoc](https://docs.google.com/spreadsheets/d/1MaZRIkmbFmi4l9iYTp5piCtDg7zeYdqTt0d8mo5oGy4/edit#gid=0)

#### Check if everything works

```bash
cd ~ && roscore & rosrun gazebo_ros gazebo
```

```bash
cd $CATKIN_WS/src/front-end/robot_frontend && npm start
```
#### From scratch
`DO NOT DO THIS UNLESS NO JETSON IS AVAILABLE TO FLASH FROM`


Download the latest [jetpack version](https://developer.nvidia.com/embedded/jetpack) 

Install the necessary tool on the host computer

Connect the jetson to a router and the host to the same through ethernet. Start jetpack and follow the instructions
Follow the instructions and, when done, install the dev environment as if you were a dev.


## Launch a simulation
Starts the simulation (main client)
```bash
roslaunch ros_client ros_client_simu.launch
```

Start the robot server with its front-end
```bash
roslaunch ros_client ros_client_server.launch
```
Then start the navigation in simulation mode
```bash
rosrun aida_navigation start --simu
```

Finally, after changing the mode to remote control on the frontend, you'll be able to control the robot on the simulation using the controler

## Use RVIZ
Rviz will show you sensor simulation and other useful information
```bash
roscd aida_navigation
cd rviz
rosrun rviz rviz -d file
```
Or use `slam_vizu`
```bash
roslaunch aida_navigation slam_vizu.launch
```
For more information, troubleshooting or getting started about each package, please visit its own README.md


## Unit testing and Integration testing
Test locally

Install Docker
    [Getting Started on Mac](https://docs.docker.com/docker-for-mac/)
    [Getting started on Linux](https://www.howtoforge.com/tutorial/docker-installation-and-usage-on-ubuntu-16.04/)

Build the image (TODO This will only work on Master) 
    docker build . -t latest

To mount the repo as a data volume and launch catkin_lint

    docker run  -v  ~/catkin_workspace/unsupervised:/home/unsupervised latest catkin_lint /home/unsupervised/ros/src -W2

To mount the repo as a data volume and launch catkin_lint


Jenkins CI (only once)
    
connect to AWS: (ask for AWS key and make sure it's read only) 

     chmod 400 unsupervised.pem
     ssh -i unsupervisedai.pem ubuntu@ec2-52-11-183-63.us-west-2.compute.amazonaws.com

Generate ssh key on jenkins user
    
    sudo su jenkins
    ssh-keygen

and add it to github
...

## Useful git tools
a GUI interface to organize your commit easily
```bash
sudo apt-get install git-gui
```

a tool to view the commits history from the terminal
```bash
sudo apt-get install tig
```

a GUI interface to view the commits history
```bash
sudo apt-get install gitk
```
A full GUI for git: [Gitkraken](https://www.gitkraken.com)


## Miscellaneous
useful links : 
[gazebo and ros simulation](http://www.generationrobots.com/en/content/75-gazebo-and-ros)


TODO: Install cartographer
TODO: add aws config
TODO: Redo comments inside .uai_config
TODO: Add troubleshooting steps