Steven Zeiler

I would like to build a system that allowed me to post micro-tasks for my official employees to work on. Smaller tasks could be used to train entry and potential employees, while established employees could have a steady stream of tasks but are not bound to any specific hours or minimum work.

The system requires access to an official people database with characters that represent each of the employees. I have been considering using salesforce.com as the primary people database because they have marvelous api access and I think learning the service will be extremely valuable.

The concept I have been mulling over for several years but was recently well pronounced by Second Life founder Philip Rosedale on my new favorite tv should Foundation http://foundation.bz/8/.

Recently I have discovered Nitrous.io which provides a hosted cloud development environment for ruby, node.js, python, and go. The environment comes pre-configured with enough software to run a public webserver, and can be accessed from any browser! This services is certainly a top candidate for teaching new students to learn web programming, as the software installation barrier to entry is almost entirely eliminated. Unlike Heroku there is almost no deploy time and the user can see how to actually run a web server rather than just sending code into the black box that is the heroku cedar stack.

To get started you need to sign up for the following accounts, which should all have a free option:

github.com => code sharing & collaboration
nitrous.io => cloud development environment
codeschool.com => cloud code learning
heroku.com => cloud app & database hosting
build.phonegap.com => mobile app builder & distributor

My strategy is to teach students entirely in the cloud in order to eliminate the most costly startup tasks such as installing software dependencies on varied platforms. This will especially help to introduce windows users to the unix shell without the myriad hassles involved in setting up a proper we b development environment.

A high-level comprehensive understanding, documentation and scripting of machine infrastructure for web applications is damn hard but is ostensibly of massive value. Institutional security is enhanced every time a software system is documented using executable system management and build scripts, whether those be at the high-level concept of chef-server or a puppet system, or more specific like the underlying shell scripts that are executed by those systems. If anyone is going to approach the infrastructure-as-code challenge they will be forced by the shear gravity of the task to approach it in a methodical fashion by scripting the most valuable institutional software assets in a decreasingly valuable order until the desired level of predictability, automation and backup is achieved or the price of moving forward become prohibitive.

Beginning with the class of software system most valuable to the company I immediately look at the database layer. The entire business of serving precise search results for classified ads is based upon the reliable availability of massive amounts of data stored in hundreds of databases using dozens of database technologies Countless business functions are served by databases and rely critically on consistent access to them. To bring the most valuable portion of data storage into a replicable and uniform system would entail the discovery of the most critical database server machines. Again even the top 80% of the most important database servers is a large set of technologies and a daunting collection to automate. Let us find the twenty perfect most valuable database machines across the various architectures and prove that we can build only those machines using scripts alone.

For many organizations targeting the most valuable databases probably means working with one mysql server or a microsoft sql server. To aim at automating the mysql system entails fir a build of all the requisite software onto the machine, then a creation of the databases, followed finally by a load of the data to bootstrap the database. Each instance of the mysql system should be separate from any application code that accesses it. Usually I predict the system would naturally dicate a topology at such scale where the mysql server software is the only process on the machine that communicates modularly over a unix socket connection with the rest of the system. The boundary of our definition of automation of the mysql layer should be here at the socket level interface.

Several pieces of configuration information are required as inputs to any mysql system and must by supplied by an engineer or hard-coded in config files. From a readable documentation perspective for managers and admins to conceptualize each piece of the automation system as a set of required inputs and predictable outputs yields the most profit and confidence. I propose we set upon automating the mysql system with a firm mysql server generation function the ultimate goal. We can do this with a variety of technologies and I think it best that we test the top strategies. Chef, puppet and capistrano all seem proper for this task, and it will probably suffice to test only the first two and perhaps attempt raw bash shell scripts. We win when we can provision a new ubuntu machine, run our script and then connect using the Mysql client and execute core application queries against the database. The next obvious win would be to connect the newly-created mysql server to a running application stack to perform integration testing before officially releasing our new business robot. Later we will document and script the various ways that our system interacts with the mysql server, such as relating to load balancers and applications servers.

Discussed often is the idea of spending a day learning how to re-build a portion of the infrastructure. I propose a more focused, specific recovery and automation challenge for the team: The scenario:

A malicious or spontaneously-generated virus has deleted all of your mysql database servers and the ‘golden’ machine images that were supposed to guarantee them. Luckily we still expect a delivery feed of new few daily data across the internet in fifteen minutes, and we still have our data loading scripts and our project’s code is intact on github. Can we take a blank copy of the Amazon Ubuntu image and have the database ready for the data feed in fourteen minutes?

There exists a fundamental difference between web servers and application servers that until recently I did not fully understand, and which needs to be clearly delineated in the students’ mental construction of the web stack and the lifecycle of a request on the internet.

Students cannot be exposed to servers initially through the entire application server stack that rails provides, but must rather learn about static files served via apache, nginx, or IIS after their introduction to http. Previously I had endeavored to introduce students to the server with the simple http server python tool, which creates a server running on the user’s local machine accessible to the local network. The exercise introduces then to the concept of ip addresses and port numbers and requesting files over http. At first everything is hosted locally, and they can discover http using the curl client tool. This approach worked well to demonstrate the most basic functionality of a client-server interaction and should remain a fundamental exercise, with several tweaks.

I am not yet certain whether I should teach http using fist the browser as the most basic http client, or using the curl tool. Curl does not require html, though neither does the browser, which can server up plain text but also images. The browser however does not allow the user to simulate POST, PUT, and DELETE actions. POST, PUT, and DELETE are necessarily actions served by the application server process while GET can be taught at first using only a web server. I do not need app-level functionality any time near the beginning of the course so both browser and curl can be used to introduce the client.

A perfect exercise to demonstrate actually deploying a web site behind a web server would be to purchase a godaddy domain with hosting. Godaddy serves your static web site using the apache web server, the most popular web server software in the world. Before jumping from the python simple http server to apache on godaddy I will explain the market for web servers and the recent history of Apache, IIS, and Nginx which collectively are run on over 90% of the world’s webservers. Although Microsoft IIS is very popular and powerful, valuable systems are built with its stack our course will not deal with IIS as it is proprietary software, restricted and licensed for a fee. Apache and Nginx on the other hand maintain free, open availability and usage for all, so we will focus primarily on them as our web server.

For the introductory period of the course I like a thick line drawn between the the system that comprises the client + web server and the system handled in the application server domain. We can reduce our scope to the study of http, client-server, web servers, GET, html, css, javascript, json, images nad files over http with both curl or the browser. Drawing the line between the web and app servers versus around the client as a primary learning sandbox enables us to explore the web in a simplified manner while maintaining its fundamental interaction of client-server over http. We will thus be able to deploy our first web app without introducing git, app servers, heroku, aws, or server-side languages. We can spend a good amount of time covering much ground inside such a restrained sandbxo before ever thinking about server-side software and architecture.

Upon examining my resume I realized that the description of what I did at each of my jobs was severely lacking. Even whitetruffle.com gives my profile only 90% completeness because my descriptions are too short, and they are admittedly vague. Since I have meant to write up the story of every one of my business ventures I might as well do a blog post for every employment on my resume and some that aren’t.

Each post should fully describe my activities at the company along with the technology stack employed. I will describe myt favorite successes and most valuable lessons attained. I can highlight how my mentality changed through each step and why I transitioned between each. My book will be much more complete once these chapters have been codified and I can extract patterns and lessons from my experience in a more concrete manner.

Now that I have decided to dedicate some time to recording the story of my professional experience I want to begin with my first salaried position where I was paid full-time to work with rails, well above the median income for the area. The venture of which I speak was both extremely valuable and mildly disappointing, as I was fired for the first and so far only time from that job.

The reason I would like to write about my time at Upstream Communications is because of the practical tools I picked up from their product offering. The studio produced websites that needed to be updated frequently with new content, calendar events, and legislative alerts. The company derived the value for their existence by building content management system backend software on the same server powering their website. Clients who don’t understand coding could then log in to their site using an administrative email and password and update the data displayed in the various lists, sections, images and videos on the page for on-the-fly customizability. The content management systems we built were very similar in functionality to wordpress but with all the flexibility and power of ruby and mysql on the server.

My responsibility was to build websites for various political campaigns with rails, mysql, html, jquery, and css, designing the site and setting up the administration backend panel. I would then deploy the website to a cloud machine running in Heroku’s webserver farm. After a month of repeatedly cranking out rails-based CMS websites I was pretty good at it, and that specific skill has been extremely valuable to me in the management of my own relational database content and that of my employers since then. I am more confident in my data when I have a slick admin interface on top of it. The admin module itself is a ruby library called rails_admin, which installs all the functionality for admin sign-in/authentication and the editing of database rows from a user-interface rather than using raw SQL. Rails admin creates a single database table for the user or admin model and users the Devise authentication gem to handle everything authentication, from password resets to access tokens to user registration.

Data integrity and trust therein is important at any business and since my experience working at Upstream Communications I have integrated the rails admin system into nearly every one of my rails applications, to great benefit.

Given that I was fired from Upstream I must examine the reasons for my dismissal and ask whether I was actually qualified for the position when I took it. Interviewing and hiring engineering candidates for specific tasks is often difficult because the hiring managers do not know how to be confident a prospect truly can deliver adequately-working software and the engineering candidate often does not fully understand the magnitude of the employers’ expectations for product delivery.

At the start of my career such misunderstandings about expectations and deliverables were too frequent, and my inexperience in communicating well with employers showed through. At Upstream I was capable of building their rails cms sites with all the javascript functionality and css design they desired but not quite at the rate the company expected in order to make profit. I do learn very quickly and perform intense research to improve my craft, but after six weeks it was obvious billing customers $150 per hour for my work was not a sustainable business model.

Despite the fact that I could not deliver results efficiently enough for the business and I should have been fairly dismissed for that along, I believe I was actually fired because I couldn’t help myself from speaking out about politics. The company was comprised of a hand-ful of unstable political junkies who at the time were often occupied worshiping Carl Rove and Mitt Romney. I teased them constantly that Romney woud lose because the Ron Paul supporters were just not going to vote at all. The day before I was fired I upset several other employees and they suggested the company was perhaps not the best place for me to be, and after later discovering that the entire organization was essentially funded by and existed for Karl Rove, I am glad I escaped with I did.

The whole experience of almost two months of full-time employment with a rockin’ salary was enough of a boost to my confidence that I soon thereafter went to the 2012 Rails Conference where I was able to take my understanding of the rails ecosystem to the next level and obtain an awesome job doing what I love. I learned it is best to avoid talking about politics at work but more importantly that if your institution is politically aligned against you it is best to sever the relationship and find a more appropriate setting. You deserve the perfect job and should never settle for spending all day every day in a miserable environment.

Recently I discussed the need for a more sound and robust strategy for storage data at Zenterp. Our media and code files represent the majority of the company’s assets and their integrity is of prime importance. After studying the industry I have definitely decied to store the company’s data on cloud file servers. Initially all of our digital assets were stored on Amazon S3, but several payment and access mishaps resulted in the accounts being locked temporarily; I had to diversify amoung cloud storage providers for security through redundancy and for research.

Amazon S3 storage is wonderful from a developer perspective because it provides code API access to everything! This is generally true for all Amazon AWS services and I love to script such services using the official ruby gem (library) provided. Alternatively one can use the marvelous “Fog” ruby library which provides a similar interface to the AWS gem, but is actually an abstraction over many storage and compute providers.

Fog allows us to upload and manage files on Rackspace Files and Google Cloud Storage using the same API as with S3. From a business perspective these three storage systems are the most interesting. I want to research these cloud providers and develop organizational best practices for media and data storage redundnancy in the cloud. AWS is most appealing because they are the leaders and have the most experience in the cloud arena, and I have done a lot of work scripting their services. Rackspace intrigues me because their infrastructure is built on top of OpenStack and a business relationship with them could help Zenterp move into building systems on open-source hardware stacks, a stated purpose for the existance of the company. Finally Google is certainly a top-notice cloud service provider and I believe it will benefit us to study using their Cloud Storage, Compute Engine, and App Engine for potential business solutions in the future. They also provide a third level of redundancy for the media as backup.

Now that I have a strategy for redundant media storage in the cloud, I will move to implement the strategy. I still need to work out a fail-safe backup on a nearby physical drive for emergencies. Once the media files are distrubuted to the three storage providers I will report back and form a strategy for content distribution to the public and with the Zenterp organization.

From time to time it really helps to write down all the things I have been studying and decide which of them should be afforded highest priority, and then alter my study patterns accordingly. Although it seems like I have several options for the most-profitable ventures in which to invest my time and energiues, there are in reality probably dozens :-P My task is to decide on which of these paths to focus now and which later.

SQL and the MySQL Database System

I have determined that in the immediate future I need to study further the sql language and the mysql software database software system. There seems to be a great need for architects who understand how to scale mysql in a cloud application environment. A stellar colleage recently recommended on and only one book to me: High Performance MySQL. I am working my way through that book now and I believe it will be of monumental value. The book is quite large so it will have to knock off two other large books I have been reading: The Linux Programming Interface and the Postresql 9 Manual. I will definitely finish The Linux Programming Interface but may not get back to the psql manual. Along the same thread of study I continue to read SQL Antipatters by Pragmatic Studios, and keep Seven Databases in Seven Weeks near the top of my queue.

On The Edge of Javascript and The Web

As database comprehension may be of primary import for the next few month I still aim to continue my study of the web ecosystem, the browser, and javascript. It is in building javascript applications targeted for chrome that I believe I will be able to construct a design and engineering company, in part because of the novelty of some of its more powerful features but mainly because of the free nature of the platform itself. I am currenly deep into a book on 3D Graphics programming in the browser called WebGL Up and Running. Limitless potential exists for creative design and user experience on the web and I want to build a company on top of said potential. More on the side of application architecture I build on and invest in two javascript application frameworks, Ember.js and Backbone.js/Marionette.js. At this stage in my experience I am comfortable developing a production-quality application using Backbone. However it is obvious that the design patterns in Marionette.js simplify app development and increase performance in many cases, so I am actively working to integrate the Marionette framework into my Backbone workflow. From an organizational standpoint Ember.js may in fact be the best way to build standardized js apps using many of the industry’s best practices. Because of the advancement of the Cordova toolchain for deploying native javascript/dom applications and the relative ubuquity of web designers and combined with my years of javascript and html5 experience, the web stack is very attractive to me from a business standpoint. I know I can successfully build a team from the ground up based on the open web stack. Given recently-available techs such as web sockets, tcp sockets in chrome, canvas, webgl, native function bindings and powerful object libraries like google maps js api and multiplied by the open-source freeness of the chrome platform the potential for exciting business opportunities in the near future is immense. Finally as a note, on book I saw and want to read is High Performance Browser Networking, as network architecture is a topmost priority in this industry.

Electronics and C++

Further down the road for me and not of immediate finanical or business impact is the study of embedded systems and programming in c/c++. For about six months I have experimented with the electronics prototyping platform arduino; I learned to proram and run a server embedded in the microcontroller that controls an LED ligh. I learned how to operate the above server over both Ethernet and Wifi. I figured out how to read data from the button status of wii nunchuck remotes and stream that data to my mac and across the network. I built live graphs of data streamed from a lightmter connected to the arduino and I controlled a small motor. Those experiments are just the beginning and my next project is to build a working, flying quadcopter from scratch. Although I would love to spend more time studying electroics, the cost of starting a business and even prototyping in the electronics sector is much high that in software. A goal of mine is to use the profits from my web/mobile software ventures to edify and electronics research lab for wireless computing, wearable computers, flying robotics, and 3D printing. In the mean time before I can commit many resources to electronics and the study thereof I can learn more c++ and read the pragmatic book Modern C++ Programming with Test-Driven Development. My long-term business strategy extends out this far conceptually, to producing electroics systems for the manufacture of micro drones and other futuristic appliances.

Algorithms & Formal Computer Science

Finally on my mind is that I want my knowledge of algorithm construction and optimization to be more advanced. I want to read the book: Introduction to Algoritms as well as finish the iTunes U Course I began from MIT that parallels the book. Also I will continue to do increasingly difficult codeeval.com challenges, but in c++ this time rather that ruby to further my understanding of that lower-level language. The study of algorithms for mathematical computation and natrual language processing will enhance my intellect in general and my possible ventures widely. I don’t know exactly when or how often I will study algorithms in a theoretical and explicit manner but I am convinced the more I do so the faster I will advance in general.

Obviously I have a lot of work to do to get where I want technically, and now that I have ennumerated chronologically my future research my studies will be more guided and effective.

It has been a while since I produced a video tutorial screencast, though I have several good ideas solicited from my network. I think since I am studying SQL a lot that the next tutorial will be an introduction to SQL and how to build a database-backed application. For the introduction I will definitely teach how to install MySQL and use tools like the mysql command line client interface and the SQLPro software package. I can demonstrate:

1. How to create a table in SQL
2. How to populate a table with data using SQL
3. How to query that table using SQL
4. How to join two tables using SQL
5. How to create a ruby Active Record object corresponding to each table
6. How to make the same queryies using ActiveRecord
7. How to manage the database schema using ActiveRecord::Migrations
8. How ActiveRecord and migrations play in the Ruby on Rails ecosystem

I want to take the students all the way up from data in a format they can understand now (such as .csv or .xls) to how to get that data into MySQL in a low-level fashion, to using the active record pattern in ruby, and finally the sugar on top by learning how to alter your sql workflow using rails.