Taken from wikipedia, software prototyping is:
While rapid application development is:
Can these same methods be applied to infrastructure? Or does infrastructure always have to be engineered? The real answer is of course (as usual per my essays) it depends. Instead of conjecturing when it might work this text will look at three examples. One where it did not work, one where it kind of worked until it went off the rails and one where it worked like a champ.
In this particular scenario I needed to learn the basics of the Perceus Cluster Management software but I did not have the hardware in the datacenter yet. What I did have was practically a super computer for a laptop. Using virtualbox I created a 2 node cluster on a private network with a headnode. Everything appeared to work well but because I was not able to prototype the same networking and certain drivers, it ended up becoming a rather pointless endeavor. Essentially the virtualized environment was just too different than the hardware environment. From one perspective some goals were realized. I had learned how to configure some of the management software. I had even figured out how to create netboot images which paid off, however, the return on time was not worth it. It took me about 1 hour to figure out how to set up the cluster and create a netboot image. It took about 3 hours to create the virtual machines, install the head node, download and install software etc. Essentially I lost 2 hours.
Using the aformentioned virtual environment I figured out how to configure Sun Grid Engine. I ran a few jobs, modified some queues and thought *this is good* When the hardware was in place, I took it one sterp further. Since I still needed to setup all of the hardware and other software I decided to get two compute nodes functional, then give access to the a few select users under the conditions that they understood this thing could blow up at anytime and we are testing. This turned out to be a good idea. The users got me a list of software they needed installed, SGE configuration todos and some environmental settings. Even though I had to rebuild the head node due to a hardware issue, I now had a list of software to build to get the users up and running when it was ready for production. I called that a win.
Although it might be obvious at first why vmware would be good for organically testing something new, it is important to keep in mind I am referring to vmware itself. Not the guests. The approach I took to building a new vmware infrastructure was by incremental levels of risk. Eventually the infrstructure was to be designed like so:
Unfortunately, I did not have the infiniband or storage nodes but I wanted to get rolling. What I decided to do was implement the systems in steps as I waited for parts and pieces to come in. The steps I took were:
From a certain point of view we essentially started testing the automobile before it was fully road ready. We prototyped as much as we could safely and even deployed certain systems into production on vmware as long as they could actually be down. Applying the rapid prototyping method worked perfectly in this scenario.
Probably what this text has covered has been done before. Most likely, however, many sysadmins don't neccessarily look at the process in this way. Some people say that adminsitrative/system programming is a completely different world than mainstream applications programming. While in some ways they are, it is clear that even building out certain types of infrastructure bear some commonalities to programming.