The Black Liszt

Storage For the Cloud

The massive movement to Cloud architectures puts new demands on systems vendors that most of them are unprepared to meet, while at the same time devaluing special features that many vendors used to differentiate their products. Nowhere has this trend been more evident than in storage.

For years, storage has had its own silo in the data center, SAN and/or NAS, with its own storage managers and administrators. They became dependent on various storage-centric features of the different vendors.

The Cloud has disrupted this comfortable island of automation.

The Cloud is all about reliable, low-cost self-service, with tremendous automation and integration. Service, capacity and performance need to be available on-demand, with no human intervention. Everything needs to be able to grow and shrink as application needs change, with a sharp eye to capacity utilization, since it’s easier than ever to switch Cloud vendors when one stumbles or is simply no longer competitive. The same observations are true of “private clouds.”

Virtualization is a key part of achieving Cloud goals, and virtualization changes the rules of the systems game. Functions that were traditionally part of storage are now performed as an integral part of operating systems and/or virtualization software, to make them more agile.

Many companies have observed that traditional, controller-centric, feature-rich SAN and NAS solutions are not appropriate for the Cloud environment. They are simply using inexpensive JBOD’s for storage and depending on massive replication by the file system to provide reliability, typically making a minimum of 3 whole copies of the data, before backups, in order to assure availability. If the alternative is an old-technology NAS or SAN, this is a smart idea, which is why its use is growing so quickly.

X-IO has a whole different approach to storage. It’s not NAS. It’s not SAN. It’s not cheap JBOD’s with a make-lots-of-copies filesystem. It’s an intelligent storage node that not only uses, but enhances the drives from one of the major OEM suppliers, Seagate. X-IO makes them better by a large margin, and it doesn’t do all the things that are no longer needed in the Cloud environment. X-IO gives you more of what you do need for Cloud, and none of what you don’t need.

The X-IO approach to storage assumes you’re smart about building your data center. You’ll take a building-block approach, with lots of well-configured servers, network and storage blocks, with a layer of software on top of it all to orchestrate it. You want each building block to be great at what it does – do a lot, cost a little, and play its role in the overall system.

In the end, storage comes down to a small set of storage components used by everyone. Rather than ignore the details of the drives and wrap them in fancy, useless (in the Cloud) packaging like everyone else, X-IO adds value, real value, to the drives themselves. This value persists as Seagate develops and releases new drives – the 2 to 5X X-IO advantage over every other storage solution will ride the waves of new drives into the future.

X-IO spent over 10 years of deep development of unique IP (the first 5 as a Seagate division). Over that time it invented and hardened algorithms and code and incorporated the experience from having thousands of units in the field over many years. The results are clear, and differentiate the X-IO storage brick approach from everyone else. Given a set of drives, X-IO will make them:

• Perform at least twice as fast, often 3-4X anyone else when near capacity
• Deliver at least twice the throughput
• Fail at less than 1% the rate of anyone else
• Not require replacement during their 5 year warranty
• Take much less space, often 30-50% less
• Require much less power, often 50% less
• Require less cooling

Finally, X-IO can incorporate SSD drives as required to achieve even better performance, though this is needed much less often than with other vendors.

In service operations, Cloud is measured on cost and SLA’s. X-IO storage is all about cost and SLA’s. X-IO is the winning choice of storage for Cloud.

Software: Comparing Waterfall and Agile

Lots of people talk about the evils of waterfall-style development. They aspire to move to something they think is better. Agile is high on most short lists for the something better. How different are waterfall and agile? Answer: not much.

Waterfall

The Waterfall model is an ordered, systematic method for determining what a computer system needs to do (the requirements) and then getting it done and into production. Like this:

The method is well-named. It really does look like a waterfall, like that big one famous for honeymoon visits on the St Lawrence River:

Above is a picture of Niagara Falls I look a little while ago, and is good for understanding software waterfalls. See the big river of water flowing from the upper right? See how everything is clear as it starts to fall? Then you see there's all the mist, making it very hard to see anything clearly at the end. Kind of like most software projects... This one gives you a good sense of the transition from clarity to mist:

Of course everyone hopes for the good outcome, for the rainbow emerging out of the mist:

But, I'm sad to say, the experience of Ms. Annie Edson Taylor comes closer to the common experience of waterfall software development:

While there is a vast array of software development philosophies, waterfall appears to be the standard against which most of them are compared; her concluding remarks saying it all: "nobody ought ever do that again."

Agile

Naturally, people look for better ways, and find lots and lots of ways that are thought to be better. It is incredible the number of software development philosophies there are. They go on and on! At least in my experience, Agile is the one I most often hear as a replacement for waterfall.

Like with all these things, people have a lot to say about Agile. There are books and books and conferences and training and certification, endlessly. Here is a summary diagram, given at roughly the same level of detail as the waterfall diagram above:

Lots of strong claims are made for Agile. It's faster, leads to better results, etc. Stuff that everyone says they want. But what are the real differences?

Comparing waterfall and agile

Take a close look at the two diagrams. Both of them start from requirements and go through design, development, test, integration and delivery. Here's the difference: with waterfall, you determine all the requirements up front and then drive through to delivery. The requirements are fixed, and you determine the time from there. In Agile, you determine a bunch of starting requirements, deliver them in a fixed time period (for example 2 to 6 weeks), and then get another set of requirements, and keep cycling until the project is done.

Waterfall: first fix the requirements, figure out the time.

Agile: fix the time periods, and then repeat until you're done.

Putting all the rhetoric aside, the difference between the two methods is simple: one determines the time from fixed requirements, and the other takes fixed time periods and fits requirements into them as appropriate. In other words, Agile is little more than a series of time-fixed waterfalls!

Remember, it's all just Process!

It's easy to get caught up in all this and forget that the most important thing isn't what makes Waterfall and Agile different -- it's how they're the same. Not exactly the same, but the same kind of thing: process!

You can build 100,000 lines of really crappy code using Agile. You can build 10,000 lines of great code that accomplishes the same thing using Waterfall. Or the other way round.

In Simple Terms

In simple terms, Waterfall is:

Do once: {Define. Design. Do. Check. Deliver.}

and Agile is:

Do until done: {Define. Design. Do. Check. Deliver.}

Conclusion

There are many good things about Agile. It's more iterative and can allow for more feedback loops than pure Waterfall. But its difference from Waterfall is easily exaggerated, which helps explain why the results in practice are so often disappointing. In the end, switching the precedence of the two key variables (requirements and time) can't make that much difference when the fundamentals of software and its Postulates are not addressed.

In storage, there is X-IO, and then there are all the others…

In normal times, when there is no major technology disruption in the market, there are two categories of storage companies.

Most storage revenue goes to the big names everyone knows (EMC, NetApp, Etc.). These companies have comprehensive storage solutions and services to meet nearly any need. Their products are solid and meet most mainstream needs. They don’t innovate much and aren’t the most cost-effective, but they work.

A good deal of attention in the storage industry goes to the hot new companies, which are all about the latest technologies (e.g. SSD) or features. They usually don’t do the old things as well as the established companies. But by focusing on the hot new thing, they often do that one thing pretty well, and so appeal to the usually tiny part of the market that feels the corresponding pain. If they get market momentum, they are usually bought by an established vendor.

This is the way it works. The established companies take most of the revenue and do little innovation. There is always a flurry of new companies trying to innovate, sometimes getting traction, and getting absorbed by the established vendors.

Then there are technology disruptions. That’s when the rules change. Suddenly the comprehensive product lines of the established vendors don’t meet the needs of the emerging landscape very well (in spite of the furious efforts of their marketing groups to claim they do), and most of the new vendors don’t get the new situation and continue to do little but exploit new devices or add features onto the existing pile.

Today’s Technically Disrupted World

That’s the situation we’re in today, with the combined technology disruptions of data centers employing virtualization, moving to the Cloud, and attempting to exploit Big Data. In addition, there is a new storage technology, flash (SSD), which vendors are scrambling to exploit. The situation is confusing for buyers and chaotic for vendors, since most vendors try to act as though nothing fundamental has changed. But it has!

The Cloud is all about reliable, low-cost self-service, with tremendous automation and integration. Service, capacity and performance need to be available on-demand, with no human intervention. Everything needs to be able to grow and shrink as application needs change, with a sharp eye to capacity utilization, SLA's and costs, since it’s easier than ever to switch Cloud vendors when one stumbles or is simply no longer competitive.

Virtualization is a key part of achieving Cloud goals, and virtualization changes the rules of the systems game. Functions that were traditionally part of storage are now performed as an integral part of operating systems and/or virtualization software, to make them more agile. This also drives the movement to software-defined networking and storage.

Big Data is the same only more so, with its emphasis on linearly scalable arrays of compute nodes and storage nodes.

In response to this massive technology disruption, many companies realize that brand-name vendors no longer make much sense, and are using inexpensive JBOD’s for storage and depending on massive replication by the file system to provide reliability, typically making a minimum of 3 whole copies of the data, before backups, in order to assure availability. If the alternative is an old-technology NAS or SAN, this is a smart idea, which is why its use is growing so quickly.

X-IO

And then there is X-IO. While X-IO is a storage company, it’s different than all the others. It was built for a vision of computing that we now call “Cloud.”

When X-IO was started about 10 years ago as the Advanced Storage Architecture division of Seagate, its goal was to build highly compact, efficient and reliable storage building blocks using Seagate HDD’s. While the rest of the storage world was ignoring the details of the devices on which it was built, piling on features and management systems that have become obsolete in the Cloud world, the ASA group was inventing the technology of the storage “brick,” now amounting to over 50 patents and a great deal of field-hardened code that delivers more of what Cloud needs than any existing system, by far.

All storage vendors, whether established or emerging, use the same drives from the same couple of leading vendors, mostly Seagate or Western Digital (WD). All of them except X-IO package them in roughly the same way and throw some features on top of them to “differentiate” themselves from the other guys who use the same disks. It’s just as though all cars had one of two different kinds of nearly-identical engines in them – each of the car vendors would try to distract you from the engine, and try to get you to appreciate how wonderful their steering wheels or cup holders were. That’s even true of NAS and SAN, which seem so different, but really have the same engines (disks) in them – it’s like one has front-wheel drive and the other rear-wheel drive, but under most conditions, their speed, fuel efficiency, acceleration and service frequency are identical.

The only storage vendor that is different is X-IO, and X-IO’s difference just happens to be on all the dimensions that matter most for the new world of Cloud, virtualization and Big Data.

X-IO’s Difference

First of all, X-IO doesn’t build feature-encrusted storage, like a “trophy car.” It’s basic storage, a storage building block or brick, ideal for plugging into nodes in a Cloud server farm under virtualization control, or a Hadoop cluster.

Second, and most important, comes from its heritage as part of Seagate. While X-IO uses the same Seagate drives that other vendors use, all the other vendors just plug the drives in and proceed to concentrate on everything but the drive. X-IO’s technology, in sharp contrast, is all about making that drive perform at its very best. You wouldn’t think there would be much that could be done. But there is! X-IO reduces the error rate of the drives so much (more than 100X) that they can be sealed in containers, which makes them take much less space, consume less power and generate less heat than the same drives in any other system. Then the X-IO software actually gets more than twice the I/O’s per second (iops) from each drive than any other vendor.

Let’s think about a car rally. Most of the cars will vary greatly in size, shape, color and gizmo’s. The X-IO car will be the plain one. Imagine them in a distance race. Most of the cars will overheat or have to stop for gas pretty often. Only the X-IO car will never overheat and get vastly better mileage than the others. Many of the other cars will break down along the way. X-IO won’t. Here’s the amazing thing: the X-IO car will cross the finish line in half the time of its nearest competitor.

Now let’s think about sending an important package. Using normal cars, you’d better send 3 identical packages by different routes to make sure it gets there. With X-IO, you only need one car, and it will get there faster than any other car, using less fuel.

In the world of Cloud, this translates into not having to buy expensive SSD drives to get performance, though X-IO has them available if you need to go even faster than X-IO normally goes. It translates into not having to over-provision to get performance. It translates into not having to store 3 or more copies of your data to assure it’s still there tomorrow. It translates into buying a half or third of the number of racks (or rows!) you would normally have to buy in order to make a given amount of data available at a given performance level. It translates into dramatically lower operating costs for those racks, which at Cloud scale and Cloud competitive pricing can be the difference between growing profitably and losing to the competition.

No other storage vendor offers these benefits. No one but X-IO.

Conclusion

The “cloud” as we know it today didn’t exist when the ASA division of Seagate started inventing the deep technology that has now matured in X-IO. But its simple mantra of getting more value out of devices was a unique quest. No vendor has equaled it, and no one is even close. As new drives are released, the X-IO advantage will persist as a multiplier on whatever Seagate ships. All the other vendors will plug Seagate drives into their systems and try to distract you, drawing your attention to “anything but” the actual characteristics of the storage – its performance, space and power use, reliability. These thing are old news in the old world of storage, but they’re the only thing that matters in the new world of Cloud. Which is why there are all the storage vendors – and then there’s X-IO.

Software Postulate: the Measure of Success

There is a Postulate of software development that, like all postulates, has huge impact on much of what goes on in software. This postulate concerns what is the measure of success in software. There are many ways to formulate it, but at heart, it's simple: success is measured by meeting expectations that have been set. Just as getting to modern physics requires changing the parallel postulate in geometry, so does getting to modern software require changing the "meet expectations" postulate in software development.

Expectations in Software

Two typical CIO's are talking with each other. They get together to share experiences because, while they don't compete with each other, their groups manage technologies of similar size and complexity.

CIO A is real happy today. "I guess they finally listened last time. They had been late once too often. I dished out a pretty blistering speech about how awful it was, and I added on a couple of threats about what was going to happen to careers if it happened again. We just had our project review meeting, and for the first time in memory, most items were green, with just a smattering of yellows and a couple reds. I breathed such a sigh of relief."

CIO B isn't so happy. "That's where I was a couple months ago. Why can't these guys just keep it up? What is it with programmers? We were mostly green, but now green projects are a fading memory. Mostly we're in the yellow and red. Yuck."

What are these guys talking about? Project Management. They're talking about whether the expectations set by their staff have been met (green) or not (yellow and red).

The green, yellow and red are at the end of a road that starts with requirements, moves on to the crucial, notoriously difficult art of estimation, and then proceeds to implementation. Green says that the estimates are being met, and yellow and red say, well, maybe not.

Introducing Absolute Measurement in Software

The CIO's get over their griping and start to compare notes on some recent projects. As it turns out, they're both building a Data Warehouse. They're in the same industry, and the projects are similar in nearly every way, at least from the outside. Common sense tells them that the internals of their projects should be pretty similar. So they compare notes.

CIO A (the happy one): "My project sounds about the same as yours. It's such a relief that we're on track. We've got a lean team of just 10 working on it (at one point I thought it might take 20 people), and we're just 6 months from the end of the 18 month project."

CIO B (the unhappy one): "What? I've got 2 people working on what sounds like the same project as yours. I'm 4 months into the project, and instead of finishing in 2 more months, they're telling me is going to stretch out 2 more weeks, a 25% overrun of the remaining time, which is why I'm so annoyed."

CIO A: "Are you kidding me? You've got just 20% of the staff with a target of 1/3 of my timeline, and you're mad? Your whole project is a rounding error compared to mine."

CIO B: "I guess my guys are doing OK after all. I just wish they could set expectations better."

On this rare occasion, the software managers confronted the reality of absolute measurement in software -- but only by chance, and only by comparing two projects to each other. They're not really even approaching absolute measurement -- if their two projects had been run equally incompetently, there would have been no surprise!

What is the Measure of Success?

What this dialog reveals is the near-universality of measuring success by comparing results to expectations. The CIO who was mad was spending very little money and getting results in a fraction of the time of his compatriot, while the CIO who was happy was doling out the money like confetti and taking the slow boat -- but since his team had started by giving him even worse estimates, he thought he was doing great.

Neither CIO was measuring the success of the projects the way most of us measure. They started with estimates. If the work was coming in better than the estimate, it was judged successful; if the work was delivered worse than the estimate, it was not successful.

This is the way it works in software. It's an unspoken assumption, a postulate that underlies nearly everything that is done in software. People don't propose alternatives, just as no one proposed an alternative to the parallel postulate in geometry for more than a thousand years. It's considered the one and only way to do things.

There are other measures of success

Groups that do an outstanding job of producing software often achieve it with a different measure of success. Just as you can optimize your work for setting expectations and meeting them, you can optimize your work to achieve maximum velocity. Estimates are less important than maximum speed. For example, there's the well-known answer to the question of how to avoid being eaten by a hungry tiger. It has nothing to do with expectations. It's simple: run faster than the other guys.

Conclusion

This is a point of deep theoretical interest, and also great practical application. It's related to building bridges in war and peace. If you're under no time or budget pressure, then maybe the meet-expectations assumption is the way you should measure your software efforts. But if you are under competitive pressure, then you might want to think about organizing your software efforts according to a different measure of success: the velocity method.

Software Development Process in Simple Terms

Software development is complicated to understand, and even more complicated to do. What's worse, developers disagree among themselves about nearly everything. Nonetheless, it's worth understanding at least the basics of what they do, confining ourselves here just to software process, ignoring (for now) the far more important software substance.

Software Terminology

Most of the talk you hear about software is about process, things like requirements, design, how and when testing should be involved, etc. There is a sea of specialized language about every aspect of software process, much of it coming from conflicting methodologies.

All of software process can be boiled down to a small number of basic, understandable things.The main steps are nearly always:

• defining what you're going to do
• how you're going to do it
• doing it
• checking it
• delivering it

What are you going to do?

Whether it's called "requirements" or "user stories," pretty much every software process starts here.

How are you going to do it?

This one amounts to the design phase. Are you going to use a DBMS? Existing libraries? Are you going to apply design patterns? Usually groups have strong preferences for these things, so the usual decisions are endorsed and people move on.

Do it

Finally! People actually do stuff!

Check it

If there were a software equivalent of the Garden of Eden, in which software happened without bugs (sin), I am unaware of it. So everyone assumes that someone (probably the other guy) screwed up, and we need to fix it.

Deliver it

Finally the software needs to get from where it's built to where it's used. The methods and destinations vary, but that's what happens in this final step.

This is all process

What I've done here, as critics would say, is "over-simplify." Given the incredible number of different software philosophies, this is understandable. Even within a philosophy, differences that seem minor to outsiders are of crucial importance to those who care about that kind of thing.

This is all just process! We're just talking about formalities. For example, the process I've described also applies to building a physical structure. The same steps apply whether you're building a simple house
or the Taj Mahal.
If essentially the same process can result in a world-wide tourist destination or a starter home, is process really the most important thing?  In other words, substance is vastly more important than process.

Nonetheless, Process still matters

If substance is so important, should process be ignored? Of course not -- having a sound process is essential. The five steps I defined above need to happen, and depending on the process, appropriate sub-steps as well. For example, unless and until you hire programmers directly from the programmer equivalent of Eden, checking is a non-negotiable requirement. That's exactly why it's important to understand software process in these extremely simple terms. It's got to happen.

But then you spend most of your time and effort on building your starter homes or your Taj Mahal. In other words, you concentrate on the substance.

Conclusion

Software development is plagued with warring methodologies and a surfeit of terminology. It's worth remembering that, in the end, it all boils down to a set of simple, understandable steps that are universal.

Process and Substance in Software Development

High among the concerns of software management are questions of organization and process. While these are reasonable concerns to have, I generally find that paying attention to substance is more productive. If you think of your organization as being like a software factory (a line of thought I generally discourage), this means you should pay more attention to the widgets that come out than the organization of the shop floor.

Process

It is easy to be totally consumed by process, organization and people. Everyone wants to know who's their boss. When there are disputes, who has the deciding vote? Many people want to know their "next step" in the organization, the path to greater responsibility, power and pay. Such concerns tend to be greater in the minds of the people on the upper part of the ladder, not to mention the top, since they usually had to work at getting where they are.

Process and organizational structure are tightly tied. Is QA a separate group with its own head? Or are there QA people as part of each small group of developers? If QA is distributed, what is the reporting structure? This is complicated by the myriad of process fashions that sweep through the industry -- there are literally dozens of them in play at any given time, things like Agile and Extreme, with Lean coming up fast.

Substance

Substance is embodied in the code that is produced. Given a set of general requirements, the substance of what is produced can differ wildly. Suppose you're extending your application to mobile. Do you use HTML 5? How do you bridge to the details of the local device? Do you write in Objective C (the native language for the Apple devices)? How much do you store locally, and how do you communicate with the servers? What about all the Android devices?

And I'm just talking about the simplest questions here. Real substance is contained in the details of how the code is written in the chosen environment. For example, the code can be pretty "straight," it can have loads of parameters, it can be layered to varying extents, it can be driven to varying extents by meta-data, etc. These choices have a huge impact on the outcome.

Process vs. Substance

Dilbert illustrates the point nicely, as he often does. In the cartoon below, the pointy-haired boss focusses, as you would expect, on process. He is concerned about dates and whether Wally has met expectations that have been set, completely ignorant of the substance.

Wally, crafty as ever, claims to have created a disastrous substance. The pointy-haired boss, unable to determine whether Wally's claims about substance are true, and unwilling to risk that they may be true, gives in.

Conclusion

Don't be Wally -- but also don't be the pointy-haired boss. Pay attention to substance. Make it your business to understand it. Your attention will provide an example to your group, telling them what's important to you. Your attention to substance will be like a chef who cares that the diners love the food that comes out of the kitchen, and does so by -- what an idea -- paying attention to the food itself.

Postulates of Software Development

A great deal of what we do in software is a direct consequence of a couple of fundamental assumptions we make: postulates of software development. Only by questioning and changing those assumptions can we bring about fundamental change in the way we build software.

Postulates or axioms are rarely discussed or thought about. We just accept them, like breathing air or walking on the ground. Changing a postulate or assumption normally results in a cascade of consequences that changes a great deal.

Geometry: The Parallel Postulate

We can understand postulates in software by seeing how they work in geometry. In Euclidean geometry, there are four fundamental postulates, and a pivotal fifth one, the parallel postulate. This is the one that says, basically, if the angle between two lines isn't exactly 180 degrees, the lines will eventually cross; otherwise, they are parallel, and never meet.

What's important about this postulate (and the others) is that all the rest of Euclidean geometry is derived from them. Given the postulates, all the theorems are implied.

For example, the famous Pythagorean Theorem is one of the many theorems whose truth grows out of the small seeds of the postulates.

In the diagram above, the theorem states that a² + b² = c².

Non-Euclidean Geometries

What if parallel lines can meet? Think it's impossible? Well, think about lines on a globe.

Lines that are parallel end up meeting -- and this is business as usual in Elliptic Geometry. What's worse, the Pythagorean Theorem does not hold in non-Euclidean geometries in general, and spherical geometry in particular.

This isn't just textbook stuff. For example, Einstein's General Theory of Relativity is based on non-Euclidean geometry. In fact, questioning the Parallel Postulate and devising ways of thinking about and describing non-Euclidean spaces was essential to the development of modern physics. So long as geometry was Euclidean and only Euclidean, progress was impossible.

The Postulates of Software

So what are the software equivalents of the Euclidean Postulates? There are few questions that are more important, because only when the foundation is questioned and changed is rational, constructive, internally consistant change possible. Only with new postulates can we derive a whole new set of theorems to define software practice. Only then is fundamental change and improvement possible.