SDN in Action: Prepare for OpenDaylight Code

SDN in Action: Prepare for OpenDaylight Code

薛國鋒     xueguofeng2011@gmail.com

In the past, CT vendors were able to develop softwares on their dedicated hardwares which need the heavy investment and had successfully fended off the competition from the outside. When the era of SDN/NFV and white-box comes, all the ICT vendors and CSPs are competing on their software ablity based on the same hardware – x86 servers. CT vendors used to foster many programmers on C/C++ for the development of real-time systems, but now they are badly short of experienced Java and Python programmers when fighting against Google, Facebook and Amazon etc.

OpenDaylight is a typical example; as the leading open-source SDN controller, it has adopted the latest software technologies from the open-source world, such  as Linux / Java / Eclipse / Maven / OSGi / Equinox / Felix / Karaf / Kafka / Raft J , which creates a steep learning curve for CT veterans like me. The transformation on knowledge structure will definitely be a long and painful journey. Today we will learn and practice some popular and open-source softwares which have become the foundation of OpenDaylight:

-          Install Java and Eclipse

-          Understand OSGi, Bundle, Equinox, Felix and Karaf

-          Build and export a bundle with Eclipse and Equinox

-          Build a Felix Java Application and install the bundle

-          Deploy Karaf and run the bundle

-          Understand Maven

-          Install and integrate Maven with Eclipse

-          Create, test and install a package with Maven and Eclipse

-          Import and use a package with Maven and Eclipse

Install Java and Eclipse

gset@ubuntu:~$ sudo apt-get update

gset@ubuntu:~$ sudo apt-get upgrade

gset@ubuntu:~$ sudo apt-get dist-upgrade

// Install Java

https://www.linode.com/docs/development/install-java-on-ubuntu-16-04/

gset@ubuntu:~$ sudo apt-get install openjdk-8-jdk

gset@ubuntu:~$ su root

root@ubuntu:/home/gset# echo 'JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64"' >> /etc/environment

gset@ubuntu:~$ source /etc/environment

// Download Eclipse - eclipse-jee-oxygen-1a-linux-gtk-x86_64.tar.gz

https://www.eclipse.org/downloads/download.php?file=/technology/epp/downloads/release/oxygen/1a/eclipse-jee-oxygen-1a-linux-gtk-x86_64.tar.gz&mirror_id=1249

// Unzip and run Eclipse

gset@ubuntu:~/Downloads$ tar -zxvf eclipse-jee-oxygen-1a-linux-gtk-x86_64.tar.gz

gset@ubuntu:~/Downloads$ cd eclipse

gset@ubuntu:~/Downloads/eclipse$ ./eclipse

Understand OSGi, Bundle, Equinox, Felix and Karaf

https://www.osgi.org/developer/architecture/

OSGi stands for Open Service Gateway initiative and it is a module system for Java. It promotes a loosely coupled collaboration mechanism between modules, which are called as services. These services are just Java interfaces. Modules publish their services in a local service registry where other modules can look them up to use them. In this way, providers and consumers are not coupled to each other, since they don't have to know about each other directly. This is the service-oriented interaction pattern of publish-find-bind. The OSGi specification originally targeted embedded devices and home services gateways, but they are ideally suited for any project interested in the principles of modularity, component-orientation, and service-orientation.

In my opinion, OSGi can be treated as a type of centralized SOA system if you don't go with the misconception that SOA is all about Web Services, while REST can be adopted for the distributed SOA based systems.

The OSGi adopts the following layered model:

Bundles – Bundles are the OSGi components made by the developers;

Services – The services layer connects bundles in a dynamic way by offering a publish-find-bind model for plain old Java objects;

Life-Cycle – The API to install, start, stop, update, and uninstall bundles;

Modules – The layer that defines how a bundle can import and export code;

Security – The layer that handles the security aspects;

Execution Environment – Defines what methods and classes are available in a specific platform.

A bundle is just a standard Java JAR whose manifest contains extra markup identifying the bundle and laying out its dependencies. As such, each bundle is fully self-describing. The OSGi architecture is highly dynamic, and bundles can come and go at any time in a session. The bundle activator allows bundles to hook the start() and stop() lifecycle events and thus perform initialization and cleanup as needed. Bundles that provide function to the rest of the system must expose the API for that function by exporting the API package. Exporting a package allows other bundles to reference the classes in the package.

OSGi-based applications need to run within an OSGi container and an OSGi container is an implementation of the OSGi core framework specification. There are many open source OSGi containers like Eclipse Equinox and Apache Felix.

Equinox which the Eclipse IDE is built upon, is a reference implementation of the OSGi core framework specification, and a set of bundles that implement various optional OSGi services and other infrastructure functions for running OSGi-based systems. Equinox has fostered the vision of Eclipse as a landscape of bundles, it is responsible for developing and delivering the OSGi framework implementation used for all of Eclipse, and as the base for all of Eclipse, it ships with all major releases of Eclipse.

Apache Felix is a community effort to implement the OSGi framework and service platform and other OSGi-related technologies under the Apache license. Both Equinox and Apache Felix implements the OSGi core specification while Equinox provides some additional features. If you write something that works on Felix, it will work on Equinox as well;If you write something for Equinox and use Equinox specific features, it might not work on Felix.

Apache Karaf is a small OSGi-based runtime which provides a lightweight container onto which various components and applications can be deployed. Apache Karaf is not another OSGi implementation, but acts like a wrapper on top of an already existing OSGi runtimes like Apache Felix or Eclipse Equinox. You can configure it to use either of these OSGi implementations as a basis. The Karaf offers some value-added services on top these runtimes, and most of these services have been designed as OSGi bundles which are getting deployed when Apache Karaf gets started. Features like central logging, manipulating configuration through commands, accessing OSGi runtime remotely using SSH, integration with OS and Apache Karaf's extensible command shell are some of the most important capabilities that an OSGi application can benefit from.

Build and export a bundle with Eclipse and Equinox

// New > Project > Plug-in Development > Plug-in Project, and  select “an OSGi framework/standard”

// For the target platform in Run Configurations, create ‘xgf_osgi_test’ and select the basic bundles

// Run as OSGi Framework - ‘xgf_osgi_test’

// xgf.activator.java

package xgf;

import org.osgi.framework.BundleActivator;

import org.osgi.framework.BundleContext;

public class Activator implements BundleActivator {

                private static BundleContext context;

                static BundleContext getContext() {

                                return context;

                }

                public void start(BundleContext bundleContext) throws Exception {

                                Activator.context = bundleContext;

                                System.out.print("Hello World, this is Richard Xue!\n");

                }

                public void stop(BundleContext bundleContext) throws Exception {

                                Activator.context = null;

                                System.out.print("Bye-bye, this is Richard Xue!\n");

                }

}

// Run as OSGi Framework – ‘xgf_osgi_test’

Hello World, this is Richard

osgi> ss

"Framework is launched."

id            State       Bundle

0              ACTIVE      org.eclipse.osgi_3.12.50.v20170928-1321

5              ACTIVE      com.xgf.osgi.bundle_1.0.0.qualifier

8              ACTIVE      org.eclipse.equinox.console_1.1.300.v20170512-2111

10           ACTIVE      org.apache.felix.gogo.command_0.10.0.v201209301215

11           ACTIVE      org.apache.felix.gogo.shell_0.10.0.v201212101605

12           ACTIVE      org.apache.felix.gogo.runtime_0.10.0.v201209301036

osgi> headers 5

Bundle headers:

 Bundle-ActivationPolicy = lazy

 Bundle-Activator = xgf.Activator

 Bundle-ManifestVersion = 2

 Bundle-Name = Bundle

 Bundle-RequiredExecutionEnvironment = JavaSE-1.8

 Bundle-SymbolicName = com.xgf.osgi.bundle

 Bundle-Vendor = XGF

 Bundle-Version = 1.0.0.qualifier

 Import-Package = org.osgi.framework;version="1.3.0"

 Manifest-Version = 1.0

osgi> bundle 5

com.xgf.osgi.bundle_1.0.0.qualifier [5]

  Id=5, Status=ACTIVE      Data Root=/home/gset/eclipse-workspace/.metadata/.plugins/org.eclipse.pde.core/xgf_osgi_test/org.eclipse.osgi/5/data

  "No registered services."

  No services in use.

  No exported packages

  Imported packages

    org.osgi.framework; version="1.8.0" <org.eclipse.osgi_3.12.50.v20170928-1321 [0]>

  No fragment bundles

  No required bundles

osgi> stop 5

Bye-bye, this is Richard

osgi> start 5

Hello World, this is Richard

osgi> exit

Really want to stop Equinox? (y/n; default=y)  y

// Export the bundle: com.xgf.osgi.bundle_1.0.0.201712091122.jar

Build a Felix Java Application and install the bundle

// Download Felix - org.apache.felix.main.distribution-5.6.10.zip

http://felix.apache.org/downloads.cgi

// New > Project > Java Project -> “FelixApp”

// Properties for “FelixApp” > Java Build Path > Default outpt folder >  “FelixApp/bin” -> “FelixApp/classes”

// Copy ‘bin’,’bundle’and ‘conf’ from ‘felix-framework-5.6.10’ to ‘FelixApp’

// For ‘bin/felix.jar’, select Build Path > Add to Build Path

// For Run Configurations, create ‘xgf_felix_app’ and put ‘org.apache.felix.main.Main’ as Main Class

// Move the bundle: com.xgf.osgi.bundle_1.0.0.201712091122.jar to FelixApp/plugins

// Run as Java Application – ‘xgf_felix_app’, and install the bundle

Dec 09, 2017 11:48:36 AM org.jline.utils.Log logr

____________________________

Welcome to Apache Felix Gogo

g! ls

bin

bundle

classes

conf

felix-cache

plugins

src

g! ls plugins

com.xgf.osgi.bundle_1.0.0.201712091122.jar

g! install file:plugins/com.xgf.osgi.bundle_1.0.0.201712091122.jar

Bundle ID: 7

g! start 7

Hello World, this is Richard Xue!

g! lb

START LEVEL 1

   ID|State      |Level|Name

    0|Active     |    0|System Bundle (5.6.10)|5.6.10

    1|Active     |    1|jansi (1.16.0)|1.16.0

    2|Active     |    1|JLine Bundle (3.5.1)|3.5.1

    3|Active     |    1|Apache Felix Bundle Repository (2.0.10)|2.0.10

    4|Active     |    1|Apache Felix Gogo Command (1.0.2)|1.0.2

    5|Active     |    1|Apache Felix Gogo JLine Shell (1.0.10)|1.0.10

    6|Active     |    1|Apache Felix Gogo Runtime (1.0.10)|1.0.10

    7|Active     |    1|xgf (1.0.0.201712091122)|1.0.0.201712091122

g! stop 7

Bye-bye, this is Richard Xue!

g!

// Update FelixApp/conf/config.properties to install the bundle automatcially

felix.auto.install.1 = file:plugins/com.xgf.osgi.bundle_1.0.0.201712091122.jar

// Run as Java Application – ‘xgf_felix_app’, and the bundle is installed automatically

____________________________

Welcome to Apache Felix Gogo

g! lb

START LEVEL 1

   ID|State      |Level|Name

    0|Active     |    0|System Bundle (5.6.10)|5.6.10

    1|Active     |    1|jansi (1.16.0)|1.16.0

    2|Active     |    1|JLine Bundle (3.5.1)|3.5.1

    3|Active     |    1|Apache Felix Bundle Repository (2.0.10)|2.0.10

    4|Active     |    1|Apache Felix Gogo Command (1.0.2)|1.0.2

    5|Active     |    1|Apache Felix Gogo JLine Shell (1.0.10)|1.0.10

    6|Active     |    1|Apache Felix Gogo Runtime (1.0.10)|1.0.10

    7|Installed  |    1|xgf (1.0.0.201712091122)|1.0.0.201712091122

g! start 7

Hello World, this is Richard Xue!

g!

Deploy Karaf and run the bundle

// Download Karaf - apache-karaf-4.2.0.M1.tar.gz

http://www.apache.org/dyn/closer.lua/karaf/4.2.0.M1/apache-karaf-4.2.0.M1.tar.gz

// Copy the bundle: com.xgf.osgi.bundle_1.0.0.201712091122.jar to apache-karaf-4.2.0.M1/deploy

// Run Karaf, and it will automatically install the bundles under the deploy directory

gset@ubuntu:~/Downloads/apache-karaf-4.2.0.M1$ ./bin/karaf

gset@ubuntu:~/Downloads/apache-karaf-4.2.0.M1$ ./bin/karaf clean

// ssh Karaf remotely

gset@ubuntu:~$ ssh -p 8101 karaf@10.0.0.150

The authenticity of host '[localhost]:8101 ([127.0.0.1]:8101)' can't be established.

RSA key fingerprint is SHA256:cesISmJHPVJdyItSsWCBtwiU0jTQLKKe0m7MzAO/WLA.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '[localhost]:8101' (RSA) to the list of known hosts.

Password authentication

Password:karaf

// Frequently-used commands for Karaf

karaf@root()> list

karaf@root()> shell:exec ls –l

karaf@root()> log:display | grep xgf

karaf@root()> shell:info

karaf@root()> la

karaf@root()> system:shutdown                                                                       

Confirm: halt instance root (yes/no): yes

karaf@root()> Bye-bye, this is Richard Xue!

Understand Maven

Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project's build, reporting and documentation from a central piece of information.Maven’s primary goal is to allow a developer to comprehend the complete state of a development effort in the shortest period of time. In order to attain this goal there are several areas of concern that Maven attempts to deal with:

-          Making the build process easy;

-          Providing a uniform build system;

-          Providing quality project information;

-          Providing guidelines for best practices development;

-          Allowing transparent migration to new features.

POM stands for "Project Object Model" and It is an XML representation of a Maven project held in a file named pom.xml. The POM contains information about the project and various configuration detail used by Maven to build the project. POM also contains the goals and plugins. While executing a task or goal, Maven looks for the POM in the current directory. It reads the POM, gets the needed configuration information, and then executes the goal.

Maven is actually a plugin execution framework where every task is actually done by plugins. Maven plugins are used to: create jar files, create war files, compile code, unit test code, create project documentation, and so on. Almost any action that you can think of performing on a project is implemented as a Maven plugin.

A repository in Maven is used to hold build artifacts and dependencies of varying types. There are strictly only two types of repositories: local and remote.

The local repository refers to a copy on your own installation that is a cache of the remote downloads, and also contains the temporary build artifacts that you have not yet released.

Remote repositories might be set up by a third party to provide their artifacts for downloading, or within your company, used to share private artifacts between development teams and for releases.

Archetype is a Maven project templating toolkit, and will help authors create Maven project templates for users, and provides users with the means to generate parameterized versions of those project templates. Archetypes are packaged up in a JAR and they consist of the archetype metadata which describes the contents of archetype, and a set of Velocity templates which make up the prototype project.

Install and integrate Maven with Eclipse

// Install Maven

gset@ubuntu:~$ sudo apt-get install maven

gset@ubuntu:~$ mvn -v

Apache Maven 3.3.9

Maven home: /usr/share/maven

Java version: 1.8.0_151, vendor: Oracle Corporation

Java home: /usr/lib/jvm/java-8-openjdk-amd64/jre

Default locale: en_US, platform encoding: UTF-8

OS name: "linux", version: "4.10.0-42-generic", arch: "amd64", family: "unix"

The settings element in the settings.xml file contains elements used to define values which configure Maven execution in various ways, like the pom.xml, but should not be bundled to any specific project, or distributed to an audience. These include values such as the local repository location, alternate remote repository servers, and authentication information.

// Create the user settings with the global settings

gset@ubuntu:~$ cd .m2

gset@ubuntu:~/.m2$ ls

repository

gset@ubuntu:~/.m2$ cp /usr/share/maven/conf/settings.xml  ./

gset@ubuntu:~/.m2$ ls -l

total 16

drwxrwxr-x 4 gset gset  4096 Dec  9 14:28 repository

-rw-r--r-- 1 gset gset 10216 Dec  9 14:54 settings.xml

// Displays a list of the platform details like system properties and environment variables.

gset@ubuntu:~$ mvn help:system

[INFO] Scanning for projects...

…

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 7.777 s

[INFO] Finished at: 2017-12-09T14:56:04-08:00

[INFO] Final Memory: 13M/126M

[INFO] ------------------------------------------------------------------------

// Integrate the newly-installed Maven with Eclipse, and abandon the embeded version

// Windows > Preference > Maven > Installations > Add 

// Add the view of Maven

// Windows > Show View > Other > Maven >  Maven Repositories

Create, test and install a package with Maven and Eclipse

// Create a Maven project – test with the archetype-quickstart 1.1, and specify the parameters

// File > New > Others >  Maven Project

// Check what the archetype-quickstart generates

gset@ubuntu:~/eclipse-workspace/test$ tree -L 2

gset@ubuntu:~/eclipse-workspace/test$ tree –l

// com.xgf.test.app.java

package com.xgf.test;

public class App

{

                public void print(String str)  // Added code

                {

                                System.out.println( "Hello World, this is " + str + "!\n" );

                }

                public static void print() // Added code

                {

                                System.out.println( "Hello World, this is Richard Xue!\n" );

                }

public static void main( String[] args )

{

System.out.println( "Hello World" );

}

}

// Test and make the package

gset@ubuntu:~/eclipse-workspace/test$ mvn test

gset@ubuntu:~/eclipse-workspace/test$ mvn package

gset@ubuntu:~/eclipse-workspace/test$ ls target -l

total 24

drwxrwxr-x 3 gset gset 4096 Dec  9 15:34 classes

drwxrwxr-x 2 gset gset 4096 Dec  9 15:34 maven-archiver

drwxrwxr-x 3 gset gset 4096 Dec  9 15:34 maven-status

drwxrwxr-x 2 gset gset 4096 Dec  9 15:34 surefire-reports

-rw-rw-r-- 1 gset gset 2197 Dec  9 15:34 test-0.0.1-SNAPSHOT.jar

drwxrwxr-x 3 gset gset 4096 Dec  9 15:34 test-classes

// Install the package to the local repository of Maven and check

gset@ubuntu:~/eclipse-workspace/test$ mvn install

gset@ubuntu:~/eclipse-workspace/test$ ls ~/.m2/repository/com/xgf/test –l

gset@ubuntu:~/eclipse-workspace/test$ ls ~/.m2/repository/com/xgf/test/1.0.0 -l

Import and use a package with Maven and Eclipse

// Create a Maven project – testclient with the archetype-quickstart 1.1, and specify the parameters

// Update pom - com.xgf.testclient.pom.xml, and add the depenency on com.xgf.test:1.0.0

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

  <modelVersion>4.0.0</modelVersion>

  <groupId>com.xgf</groupId>

  <artifactId>testclient</artifactId>

  <version>0.0.1-SNAPSHOT</version>

  <packaging>jar</packaging>

  <name>testclient</name>

  <url>http://maven.apache.org</url>

  <properties>

    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

  </properties>

  <dependencies>

    <dependency>

      <groupId>junit</groupId>

      <artifactId>junit</artifactId>

      <version>3.8.1</version>

      <scope>test</scope>

    </dependency>

    <dependency>

      <groupId>com.xgf</groupId>

      <artifactId>test</artifactId>

      <version>1.0.0</version>

      <scope>runtime</scope>

    </dependency>

  </dependencies>

</project>

gset@ubuntu:~/eclipse-workspace/testclient$ mvn dependency:tree

// com.xgf.testclient.app.java

package com.xgf.testclient;

public class App

{

    public static void main( String[] args )

    {

        com.xgf.test.App temp = new com.xgf.test.App();   // Added code

        temp.print("Robert");  // Added code

        com.xgf.test.App.print();  // Added code

    }

}

// Run as Java Application  - com.xgf.testclient.App.Main