Introduction
Iron.io has a product called IronWorker which provides a task oriented Linux container that you can run your programs inside. If you are not sure what I mean, think of this as having a temporary Linux virtual machine instantly available for your personal but short term use. IronWorker allows you to load your binaries, code files, support files, shells scripts and just about anything else you may need to run your program in the container. You specify a single task to execute, such as running a shell script or a binary and IronWorker will perform that task when requested. Once the task is complete, IronWorker will tear down the container as if it never existed.

If you are developing in Windows or on the Mac and plan to load and run pre-built binaries in IronWorker, they must be built for Linux. If that could be a problem don’t despair, you have options.

You can create an Ubuntu VM so you can build the Linux binaries you need. This is what I do. I don’t develop in my Ubuntu VM, I use it as a testing and staging area. Your second option is to build your program inside of the IronWorker container and then execute it. You have a lot of flexibility to do what you need with IronWorker.

To Install or Not to Install Ubuntu
I highly recommend that you buy VMWare Fusion or Parallels and create an Ubuntu VM. Most of the cloud based environments you may choose to use for your testing and production environments will be running Linux. It helps to stage and test things before uploading code to these environments. I use VMWare Fusion and I love the product. VMWare Fusion will cost around $60 USD and Ubuntu is FREE. I run Ubuntu 12.04 LTS but version 13.04 is now available.

http://www.vmware.com/products/fusion/
http://www.ubuntu.com/download/desktop

In this post I will be using my Mac for everything. So if you are running Windows you should be able to follow along. Once you see how to build your applications inside the IronWorker container, you will know how to load and run Linux binaries. For those who are interested in setting up or using Ubuntu, read the next section. If you want to continue in your Windows or Mac environment, go to the Installing Ruby section.

Setting Up Your Terminal In Ubuntu
If you decided to go ahead and install an Ubuntu VM or you already have one, awesome. Here are a few things you will want to do so you can use IronWorker.


The first Icon in the Launcher bar is the Dash Home Icon. Select this and perform a search for the Terminal application.

You will see several terminal applications show up. Select the Terminal program, which will launch a Terminal Session. Now back in Launcher you will see an icon for the Terminal session you started. Right Click on the Terminal Icon and select Lock To Launcher.

One last step, we need to change a configuration option for Terminal:


Make sure the Terminal session is the active program and move your mouse to the top of the screen. The menu for Terminal will appear. Under Edit choose Profile Preferences. Then select the Title and Command tab and check the Run command as a login shell. We need this for our Ruby programs to work properly in our Terminal sessions.

Installing Ruby
Before we can start using IronWorker we need the latest version of Ruby installed. IronWorker provides a Ruby based command line tool that allows us to upload IronWorker tasks for the different projects we create.

Window Users

In Windows use this website to install the latest version of Ruby:

http://rubyinstaller.org/

Once you are done skip to the next section called Installing IronWorker.

Mac and Linux Users

On the Mac and Linux I have found the best way to install and manage Ruby is with the Ruby Version Manager (RVM):

https://rvm.io/rvm/install

Perform these steps on both your Mac and Linux operating systems. Open a Terminal session and run this command:


Use the backslash when running the command. This prevents misbehaving if you have aliased it with configuration in your ~/.curlrc file.

That command should download the latest version of Ruby and make it the default version. Before moving on, close your Terminal session and start a new one. Check that everything is good by running this command:


This should be the output:


There may be a newer version of Ruby by the time you are reading this. But as long as the version is greater than or equal to 2.0.0 and has the =* operator in front, you will be all set.

If you are not sure if you have the correct default version set and want to make absolutely sure, run this command:


Just make sure you specify the version correctly.

Just for your information, the rvm tool and the different versions of Ruby you install get placed in a folder called .rvm inside your $HOME folder. It will be a hidden folder. To see that is does exist, run the following command in your Terminal session:


This should display the name of the directory back in the Terminal session.

Installing IronWorker
With Ruby properly installed we can install the IronWorker tool. This tool is a Ruby program that allows us to create tasks by uploading our programs and support files into IronWorker.

To install the tool we must install the IronWorker Ruby gem. Gem's are Ruby packages that get installed under the default version of Ruby. This is why I stressed we have the right default version set.

Run the following command in the Terminal session:


If everything installs properly, the output should end like this:


As a final test to make sure everything is setup correctly, run the IronWorker program:


If everything is working properly you should get the following output:


Now we have our environment setup to talk with the IronWorker platform. Make sure you do this in all of your environments.

The Test Program
I have built a test application that we are going to run in IronWorker. To download the code and the IronWorker support files, run the following commands:


This will copy, build and install the code into the example folder under $HOME. The program has been written to test a few things about the IronWorker Linux container environment. Let's review the code for the program first and test it locally.

For IronWorker to be really effective you want to build programs that perform a specific task. The program should be designed to be started on demand or on a schedule, run and then be terminated. You don't want to build programs that run for long periods of time. The test program runs for 60 seconds before it terminates.

There were two things I wanted to know about IronWorker that the program tests. First, I wanted to know how logging worked. Second, I wanted to know if the program would receive OS signals when I requested the running program to be killed.

With all that said let's review the code we are going to run.

Here is the logging code which can be found in the helper package:


There is nothing fancy here, just an abstraction layer so I change out the logging mechanism if this doesn't work.

Here is the controller code that manages the starting and termination of the program:


I like to shutdown my applications gracefully, so if I could receive an OS signal on a kill request, that would be fantastic. I am not a channel guru and I am sure there are better ways to accomplish this. I welcome any suggestions. For now this is what we got.

The function creates three channels. The first channel is used to receive signals from the OS. The second channel is used by the Go routine that is performing the program logic to signal when it is done. The third channel is used by the controller to signal the Go routine to terminate early if necessary.

The program.DoWork function is started as a Go routine and then the controller waits for either the OS to signal or the running Go routine to signal it is done. If the OS signals to terminate, then the controller uses the shutdown channel and waits for the Go routine to respond. Then everything shuts down gracefully.

Here is the code for the Go routine that is simulating the work:


The DoWork function prints a message to the log every 250 milliseconds 240 times. This gives us a minute of work that must be performed. After each write log call, the function checks if the shutdown channel has been signaled. If it has, the function terminates immediately.

Just to have a complete code sample in the post, here is the main function:


In your native environment, mine being the Mac, download the code and let Go tool build and install the application.


When you run the program, let it run to completion. You should see the following output:


The program started and terminated successfully. So the Controller logic is working. This time let's kill the program early by hitting <Ctrl> C after it starts:


As soon as I hit <Ctrl> C the OS signaled the program with the syscall.SIGINT message. That caused the Controller to signal the running program to shutdown and the program terminated gracefully.

Configuring IronWork
This is the documentation for using IronWorker:

http://dev.iron.io/worker/

I am going to walk you through the process for the test application we have. Login to your Iron.io account and select Projects from the top menu:


Enter Test into the text box and hit the Create New Project button. That should send you to the following screen:


Select the Key icon which is where you will find the Credentials for this project. We need these credentials to create a special file called iron.json. This file will be required by the IronWorker Ruby program to load our tasks into this project.

In our Terminal session let's move to the folder where the IronWorker script files are located. I want to work with the buildandrun scripts:


You should see the following files:


You will find a iron.json file in the folder. Edit the file and put in your credentials:


The next file we want to look at is the .worker file. Here is the documentation for .worker files:

http://dev.iron.io/worker/reference/dotworker/

Here is what the buildandrun.worker file looks like:


The buildandrun.worker file is telling the IronWorker Ruby program to upload and execute the buildandrun.sh file. This is the only file that will be placed into the IronWork container.  Here is what the buildandrun.sh file looks like:


This script tests to see if the ironworker test application already exists. If it doesn't, it then proceeds to download the latest Linux binary package for Go and builds the program using the Go tool. Once the build and install is complete, the script executes the program.

You will notice these lines have been commented out in the shell script:


If you have a repository that requires authentication you can use this technique. This calls git clone the same way the Go tool does, so everything is copied to the right directory structure. If your code references other libraries, you will need to do that manually.  If you run the Go Get command with the [-x] option, you can see all the calls the Go tool issues. Just copy what you need and add it to your shell script.

IronWorker does have a copy of the Linux binary package for Go version 1.0.2 already pre-configured in every IronWorker container. The script installs the latest version to show you how that can be accomplished if the version you need is not the one installed. The technique can also be used to install other packages that might be required.

If you want to build the code every time the task runs, you could be smarter and run go version first. If the right version is not already available then you can download the version of Go you need:


Loading IronWork With A Task
We have everything we need to load and run our first task for the Test project. In your Terminal session run the following command:


If everything is successful you should see the following output:


Go back to the Iron.io website and let's see if our new task is there. Select Projects again from the main menu and select the Worker button to the right of your Test project.


Select the Tasks tab and you should see the buildandrun task we just uploaded. Select the task and you should see the following screen:



There is a big grey button that says Queue a Task. Let's hit that button to run our task.

This dialog box will popup. Use all the defaults and hit the Queue Task button.

This will queue the task and then it should start running.
Once you hit the queue button the screen should change:



The task will start in the queued state, then running and finally complete. Once the task is done, click on the Log link.



You will see the shell script did everything perfectly. It downloaded Go and successfully built the program. Then it started executing it.

Let's run the program again and check two things. First, let's see if the container is saved and the download of Go does not have to occur again. Second, let's kill the program early and see if it receives any OS signals.

Hit the Queue a Task button again and after several seconds let's kill it:




You can see that I killed the task 21 seconds into the run. When I look at the logs I am a bit disappointed. First, the task downloaded Go again. This means I have a new and clean IronWorker container every time the task runs. Second, the program did not receive any OS signals when I issued the kill. It appears the IronWork container is forced to die and the program gets no OS notifications.

It is not the end of the world, just something that is good to know. Based on this new information it seems we want to load binaries into the IronWorker container when we can. This way we don't need to spend time downloading things that can be pre-compiled. However, I was able to use IronWorker from my Mac environment which is a real plus.

On the other hand, having Go build and install your program every time could be huge. If you upload code changes to your repository you don't need to upload a new revision of the task. The Go tool will pull down the latest code, build it and run the program. Then again, that could cause problems.

At the end of the day you need to decide what is best for your different scenarios. What's awesome is that IronWorker gives you all the flexibility you need to make it work.

Working With Builder Tasks
Builder Tasks are a hidden gem within IronWorker. A builder task allows you to build your code and generate a task that you will use for your application inside of IronWorker. This is really the best of both worlds because you don't need to download Go and build the code every time the task runs. You can do your build once. Your application task runs immediately because it is always ready to go with the binaries and other support files it needs.

Go back into the scripts folder and find the sub-folder called buildertask. Let's quickly run through the files and see how this works.

The task-builder.sh file is the script that knows how to build our code.


The code downloads Go and then uses the Go tool to build the program. At the end of the script we copy the binary that the Go tool built to the IronWorker staging area. Anything you copy to this folder will be placed into our new application task.

The task.sh file is the script that is executed by the application task.


In our case we only need to run the binary. Remember the binary is being created by the task-builder.sh script file.

The task.worker file performs all the magic:


The worker file tells IronWorker that our application task is a binary and to load and run the task.sh script. Next we have the build command. This tells IronWorker to perform a remote build by exexuting the task-builder.sh script. The file command pulls the task-builder.sh file into the builder task so it can executed remotely.

Let's navigate to the buildertask folder and try all this out:


We need to edit the iron.json file with the credentials again. Once you do that run the following command:


This time the IronWorker will take a bit longer to run. It will be performing a remote build and we must wait until it is complete. Once everything is done you should see the following:


Now let's switch to the Iron.io website and see what we have. Go back to the Tasks tab and you should see two new tasks:


You will notice the task::builder task has already been executed. This is the remote build that was being performed. Let's look at the logs. You will see that the Go binary package for Linux was downloaded and the project was also downloaded and built. Look at the last two lines in the log:


This is where we copied the binary to the staging folder __build__. We didn't have any errors or problems coping the final binary.

Now we can try running the application task. Select task from the Task list and queue the job. It should start right up and run for a minute. Once it is done let's look at the log:



When you look at the log you can see the program starts running. There is no downloading of Go or any other build work.

Using a builder task is a great way to have IronWorker stage and build the code for us. If you change the code and need to perform another build you must run the IronWorker program again. This will create new revisions of both the builder and application tasks. You can't rerun the Builder task manually.

IronWork and PaperTrail
Looking at the logs in IronWork is a real convenience but you will want to use a third party system to manage your logging. I love PaperTrail and the IronWorker integration is seamless.

Go to PaperTrail and create a free account:

https://papertrailapp.com/

After you get your account and login, go to the Dashboard.


Find the Create Group button and create a new group:


Click the Save button at the bottom. Now on the Dashboard you should have your new group:


Go to the Account options.


Create a Log Destination for your IronWorker Test project. Click on the Create log destination button:


Click on the Edit Settings button:


Make sure you accept logs from unrecognized systems and select the group you just created. Then hit the Update button.

Now copy the destination url and go back to your IronWorker Test project. Select the settings icon:


Take the PaperTrail url and enter it into the Logger URL text box using udp as the protocol. Click the Update button and then click on the Worker button again and find the buildandrun task. The udp must be lowercase or your task will fail.

Queue the task one last time and let it run to completion. As the task is running, go back to the PaperTrail website. Go to the Dashboard and hit the refresh button on the browser:


You should start seeing events coming into your group.  Click on the All events drop down on the right and you will see the logs in PaperTrail.



Conclusion
I just scratched the surface with how you can use IronWorker to run your applications. This is a really flexible environment with really no restrictions. You have access to download and install any packages you need, access to the local disk and integration to PaperTrail and a few other logging systems.

Though you don't need a Linux VM to use IronWorker, you may want to consider having one so you can stage and load your binary programs directly. Again, you have the flexibility to use IronWorker as you see fit.

I hope you try out the service. Use my application or build your own. Post a comment about your experience and anything new your learn. I plan on using IronWorker for two projects I am working on and expect only great things.