Air Force and IBM partner to prove Cloud Computing works for Defense and Intelligence services

One of the top concerns about Cloud Computing is security of the data in the cloud.  IBM has a press announcement on the partnership here.

U.S. Air Force Selects IBM to Design and Demonstrate Mission-Oriented Cloud Architecture for Cyber Security

Cloud model will introduce advanced cyber security and analytics technologies capable of protecting sensitive national data

ARMONK, N.Y. - 04 Feb 2010: The U.S. Air Force has awarded IBM (NYSE:IBM) a contract to design and demonstrate a secure cloud computing infrastructure capable of supporting defense and intelligence networks. The ten-month project will introduce advanced cyber security and analytics technologies developed by IBM Research into the cloud architecture.

There are press articles too.

CNet News

Air Force taps IBM for secure cloud

by Lance Whitney

IBM has a tall order from the U.S. Air Force--create a cloud network that can protect national defense and military data.

Big Blue announced Thursday a contract from the Air Force to design and demonstrate a cloud computing environment for the USAF's network of nine command centers, 100 military bases, and 700,000 personnel around the world.

The challenge for IBM will be to develop a cloud that can not only support such a massive network, but also meet the strict security standards of the Air Force and the U.S. government. The project will call on the company to use advanced cybersecurity technologies that have been developed at IBM Research.

and Government Computer News.

What I find interesting is how few authors reference the IBM press release.  The goal of the project is a technical demonstration.

"Our goal is to demonstrate how cloud computing can be a tool to enable our Air Force to manage, monitor and secure the information flowing through our network," said Lieutenant General William Lord, Chief Information Officer and Chief, Warfighting Integration, for the U.S. Air Force. "We examined the expertise of IBM's commercial performance in cloud computing and asked them to develop an architecture that could lead to improved performance within the Air Force environment to improve all operational, analytical and security capabilities."

Which is cut and pasted into the CNet news article as well.

On the other hand, there are some good insights by Larry Dignan on his ZDnet blog.

What’s in it for IBM? Cloud computing has a lot of interest, but security remains a worry for many IT buyers. If Big Blue can demonstrate cloud-based cyber security technologies that’s good enough for the military it would allay a lot of those worries.

The advanced cyber security and analytics technologies that will be used in the Air Force project were developed by IBM Research (statement).

According to IBM the project will show a cloud computing architecture that can support large networks and meet the government’s security guidelines. The Air Force network almost 100 bases and 700,000 active military personnel.

and Larry continues on the key concepts of what will be shown.  Models!!! yea!

  • The model will include autonomic computing;
  • Dashboards will monitor the health of the network second-by-second;
  • If Air Force personnel doesn’t shift to a “prevention environment” in a cyber attack the cloud will have automated services to lock the network down.
  • Read more

    Is Elastra one of Amazon’s Cloud Computing infrastructure tools? An awesome PDF to understand a better approach to infrastructure management

    I plan on having a meeting with Elastra next week when I am in the bay area.  I wrote about their tools last week.

    Elastra’s Cloud Computing Application Infrastructure = Green IT with a Model approach

    Elastra connects the power use in the data center to the application architects and deployment decision makers.

    Plan Composer function lets customers set their own policies based on application needs and specific power metrics (such as wattage, PUE, number of cores, etc.). Therefore, if an application requires 4GB of RAM and two cores for optimal performance, and if the customer is concerned with straight wattage, Elastra’s product will automatically route it to the lowest-power 4GB, dual-core virtual machine available.

    Gigaom has a post on Elastra’s Cloud Computing infrastructure addressing greener services.

    Elastra Makes Its Cloud Even Greener

    By Derrick Harris Jan. 12, 2010, 2:51pm 1 Comment

    0 0 33

     

    Checking out the investors, look what I found.

    Amazon

    Amazon.com, Inc.

    (NASDAQ:AMZN), a Fortune 500 company based in Seattle, opened on the World Wide Web in July 1995 and today offers Earth's Biggest Selection. Amazon.com, Inc. seeks to be Earth's most customer-centric company, where customers can find and discover anything they might want to buy online, and endeavors to offer its customers the lowest possible prices. Amazon.com and other sellers offer millions of unique new, refurbished and used items in categories such as books, movies, music & games, digital downloads, electronics & computers, home & garden, toys, kids & baby, grocery, apparel, shoes & jewelry, health & beauty, sports & outdoors, and tools, auto & industrial.

    Amazon Web Services provides Amazon's developer customers with access to in-the-cloud infrastructure services based on Amazon's own back-end technology platform, which developers can use to enable virtually any type of business. Examples of the services offered by Amazon Web Services are Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service (Amazon S3), Amazon SimpleDB, Amazon Simple Queue Service (Amazon SQS), Amazon Flexible Payments Service (Amazon FPS), and Amazon Mechanical Turk. www.amazon.com

    Then digging more I found this architecture PDF by Stuart Charlton.

    I like this picture from his personal site.

    better han the corporate one from Elastra.

    Stuart Charlton

    Back to his PDF.  Much of the information in the pdf is on the Elastra technology site, but I found it easier to read the PDF to understand Stu’s thinking.

    The introduction targets the use of Elastra for the architect, stating the problems.

    Reference Architecture Introduction

    In today’s age of on-demand access to applications, compute, storage, and networks, modern IT applications and service management has many complications:

    • Applications can be deployed across organizationally & geographically distributed data centers. The technology in these data centers, from virtualization platforms, to host, storage, and network infrastructure, is typically heterogeneous, and not necessarily managed with uniform policies and interfaces.
    • The performance, scalability, and availability characteristics of an application are due to a complex combination of design and operational decisions. The greatest impacts on these factors are due to decisions in the architecture and development of the application, before configuring the data center infrastructure.
    • Application and infrastructure management is complex and inter-disciplinary. It’s unlikely a system can be diagnosed and maintained by one person to keeping the system design & configuration in their head. Application design, administration, and management typically is a collaborative activity across specialists; there is no “one-size fits all” design tool, management tool or application platform.

    The design goals are right on.

    Three design goals for an end-to-end cloud design approach include:

    Separated Applications from Infrastructure, through modeling the application in terms of its architecture and infrastructure requirements, without tying the application to a specific set of underlying infrastructure

    Enabling Computer-Assisted Modeling and Control Automation, provided by a set of control agents and user-guided by graphical design tools. This could help IT architects and operators determine design constraints on the application, match the design to the underlying infrastructure, and enable goal-driven automation to deploy, scale, or recover their IT systems on demand.

    Explicit Collaboration To Enact Changes, through models that codify, relate and analyze the constraints and preferences that are appropriate to stakeholders across enterprise IT: from architects and developers, through operators, administrators, and managers.

    The document has many great ideas including the use of models.

    Declarative models are useful ways to drive complexity out of IT application design and configuration, in favor of more concise statements of intent. Given a declaration of preferences or constraints, an IT management system can compose multiple models together much more effectively than if the models were predominantly procedural, and also formally verify for conflicts or mistakes. On the other hand, not everything can be declarative; at some point, procedures are usually required to specify the “last mile” of provision, installation, or configuration.

    Here is a diagram showing VMware Virtual Center (Private Cloud Inventory)  and Amazon EC2/EBS (Public Cloud Inventory).

    image

    image

    Read more

    Elastra’s Cloud Computing Application Infrastructure = Green IT with a Model approach

    Elastra connects the power use in the data center to the application architects and deployment decision makers.

    Plan Composer function lets customers set their own policies based on application needs and specific power metrics (such as wattage, PUE, number of cores, etc.). Therefore, if an application requires 4GB of RAM and two cores for optimal performance, and if the customer is concerned with straight wattage, Elastra’s product will automatically route it to the lowest-power 4GB, dual-core virtual machine available.

    Gigaom has a post on Elastra’s Cloud Computing infrastructure addressing greener services.

    Elastra Makes Its Cloud Even Greener

    By Derrick Harris Jan. 12, 2010, 2:51pm 1 Comment

    0 0 33

    Elastra has incorporated energy efficiency intelligence into its Cloud Server solution, allowing customers to define which efficiency metrics are important to them and then rely on the software to route each application to the optimal resources with their internal cloud environments. Elastra’s efforts are just the latest in a growing trend toward saving data center costs by using the least possible amount of power to accomplish any given task. Especially in the internal cloud space, power management capabilities are becoming a must-have, with vendors from Appistry to VMware offering tools to migrate workloads dynamically and power down unneeded servers.

    Digging into the press release I found Elestra uses a modeling approach.

    Elastra accomplishes this through two technologies available in the product. The first technology is the ECML and EDML semantic modeling languages. ECML is a language used to describe an application (software, requirements, and policies) and EDML is used to describe the resources (virtual machines, storage, and network) available in a data center. These languages can be easily extended to enhance the definition of the applications and resources.

    These modeling languages coupled with the Plan-Composer in the Elastra Cloud Server enables users to synthesize a plan for execution. The Plan-Composer analyzes the proposed application designs (expressed thru ECML) and data center resources (expressed thru EDML), comparing them against a library of actions and outcomes. It then generates a plan based on the energy efficiency policies of the organization that can be executed by the Cloud Server against a customer’s infrastructure.

    The cool part is Elestra uses OWL and RDF to support their modeling approach.

    Elastic Modeling Languages
















    The Elastic Modeling Languages are a set of modular languages, defined in OWL v2, that express the end-to-end design requirements, control and operational specifications, and data centre resources & configurations required to enable automated application deployment & management.

    While the foundation of the modeling languages is in OWL and RDF, developers can interoperate with the Elastra Cloud Server through its RESTful interfaces; all functions available to the Elastra Workbench are available through this interface, which are based on Atom collections and serialized JSON, XML, or RDF (XML or Turtle) entries.

    Declarative models are useful ways to drive complexity out of IT application design and configuration, in favor of more concise statements of intent. Given a declaration of preferences or constraints, an IT management system can compose multiple models together much more effectively than if the models were predominantly procedural, and also formally verify for conflicts or mistakes. On the other hand, not everything can be declarative; at some point, procedures are usually required to specify the “last mile” of provision, installation, or configuration.

    Read more

    Google’s Secret to efficient Data Center design – ability to predict performance

    DataCenterKnowledge has a post on Google’s (Public, NASDAQ:GOOG) future envisioning 10 million servers.

    Google Envisions 10 Million Servers

    October 20th, 2009 : Rich Miller

    Google never says how many servers are running in its data centers. But a recent presentation by a Google engineer shows that the company is preparing to manage as many as 10 million servers in the future.

    Google’s Jeff Dean was one of the keynote speakers at an ACM workshop on large-scale computing systems, and discussed some of the technical details of the company’s mighty infrastructure, which is spread across dozens of data centers around the world.

    In his presentation (link via James Hamilton), Dean also discussed a new storage and computation system called Spanner, which will seek to automate management of Google services across multiple data centers. That includes automated allocation of resources across “entire fleets of machines.”

    Going to Jeff Dean’s presentation, I found a Google secret.

    image

    Designs, Lessons and Advice from Building Large
    Distributed Systems

    Designing Efficient Systems
    Given a basic problem definition, how do you choose the "best" solution?
    • Best could be simplest, highest performance, easiest to extend, etc.
    Important skill: ability to estimate performance of a system design
    – without actually having to build it!

    What is Google’s assumption of where computing is going?

    image

    Thinking like an information factory Google describes the machinery as servers, racks, and clusters.  This approach supports the idea of information production.  Google introduces the idea of data centers being like a computer, but I find a more accurate analogy is to think of data centers as information factories.  IT equipment are the machines in the factory, consuming large amounts of electricity for power and cooling the IT load.

     image

    Located in a data center like Dalles, OR

    image

    With all that equipment things must break.  And, yes they do.

    Reliability & Availability
    • Things will crash. Deal with it!
    – Assume you could start with super reliable servers (MTBF of 30 years)
    – Build computing system with 10 thousand of those
    – Watch one fail per day
    • Fault-tolerant software is inevitable
    • Typical yearly flakiness metrics
    – 1-5% of your disk drives will die
    – Servers will crash at least twice (2-4% failure rate)

    The Joys of Real Hardware
    Typical first year for a new cluster:
    ~0.5 overheating (power down most machines in <5 mins, ~1-2 days to recover)
    ~1 PDU failure (~500-1000 machines suddenly disappear, ~6 hours to come back)
    ~1 rack-move (plenty of warning, ~500-1000 machines powered down, ~6 hours)
    ~1 network rewiring (rolling ~5% of machines down over 2-day span)
    ~20 rack failures (40-80 machines instantly disappear, 1-6 hours to get back)
    ~5 racks go wonky (40-80 machines see 50% packetloss)
    ~8 network maintenances (4 might cause ~30-minute random connectivity losses)
    ~12 router reloads (takes out DNS and external vips for a couple minutes)
    ~3 router failures (have to immediately pull traffic for an hour)
    ~dozens of minor 30-second blips for dns
    ~1000 individual machine failures
    ~thousands of hard drive failures
    slow disks, bad memory, misconfigured machines, flaky machines, etc.
    Long distance links: wild dogs, sharks, dead horses, drunken hunters, etc.

    image

    Monitoring is how you know your estimates are correct.

    Add Sufficient Monitoring/Status/Debugging Hooks
    All our servers:
    • Export HTML-based status pages for easy diagnosis
    • Export a collection of key-value pairs via a standard interface
    – monitoring systems periodically collect this from running servers
    • RPC subsystem collects sample of all requests, all error requests, all
    requests >0.0s, >0.05s, >0.1s, >0.5s, >1s, etc.
    • Support low-overhead online profiling
    – cpu profiling
    – memory profiling
    – lock contention profiling
    If your system is slow or misbehaving, can you figure out why?

    Many people have quoted the idea “you can’t manage what you don’t measure.”  But a more advanced concept that Google discusses is “If you don’t know what’s going on, you can’t do
    decent back-of-the-envelope calculations!”

    Know Your Basic Building Blocks
    Core language libraries, basic data structures,
    protocol buffers, GFS, BigTable,
    indexing systems, MySQL, MapReduce, …
    Not just their interfaces, but understand their
    implementations (at least at a high level)
    If you don’t know what’s going on, you can’t do
    decent back-of-the-envelope calculations!

    This ideas being discussed are by a software architect, but the idea applies just as much to data center design.  And, the benefit Google has it has all of IT and development thinking this way.

    image

    And here is another secret to great design.  Say No to features.  But what the data center design industry wants to do is to get you to say yes to everything, because it makes the data center building more expensive increasing profits.

    image

    So what is the big design problem Google is working on?

    image

    Jeff Dean did a great job of putting a lot of good ideas in his presentation, and it was nice Google let him present some secrets we could all learn from.

    Read more

    Gartner says companies must implement a Pattern-Based Strategy

    In my day job, I help clients be innovative leaders, constantly looking for what it takes to be better than the rest. Gartner recently has announced a new initiative called Pattern-Based Strategy.

    It is a pleasant surprise to have Gartner’s nine analysts come to a recommendation I’ve been using for over five years in IT infrastructure.

    Introducing Pattern-Based Strategy

    7 August 2009

    Yvonne Genovese Valentin T. Sribar Stephen Prentice Betsy Burton Tom Austin Nigel Rayner Jamie Popkin Michael Smith David Newman

    The environment after the recession means business leaders must be more proactive in seeking patterns from conventional and unconventional sources that can positively or negatively impact strategy or operations, and set up a consistent and repeatable response by adjusting business patterns.

    One of the best groups I worked with at Microsoft and still have many friends in is the Patterns & Practices group, and I still have regular discussions of how Data Centers and IT could/should be using a patterns-based approach.

    You’ve probably guessed from the first half our name that we’re rather enthusiastic about design patterns.  Design patterns describe solutions to common issues that occur in application design and development. A large part of what we do involves identifying these common issues and figuring out solutions to them that can be used across different applications or scenarios. Once we have the patterns, we typically package them up in what we call an application block.

    Software people have been some of the early adopters of patterns, but the history of patterns comes from Christopher Alexander, a building architect.

    A pattern must explain why a particular situation causes problems, and why the proposed solution is considered a good one. Christopher Alexander describes common design problems as arising from "conflicting forces" -- such as the conflict between wanting a room to be sunny and wanting it not to overheat on summer afternoons. A pattern would not tell the designer how many windows to put in the room; instead, it would propose a set of values to guide the designer toward a decision that is best for their particular application. Alexander, for example, suggests that enough windows should be included to direct light all around the room. He considers this a good solution because he believes it increases the enjoyment of the room by its occupants. Other authors might come to different conclusions, if they place higher value on heating costs, or material costs. These values, used by the pattern's author to determine which solution is "best", must also be documented within the pattern.

    A pattern must also explain when it is applicable. Since two houses may be very different from one another, a design pattern for houses must be broad enough to apply to both of them, but not so vague that it doesn't help the designer make decisions. The range of situations in which a pattern can be used is called its context. Some examples might be "all houses", "all two-story houses", or "all places where people spend time." The context must be documented within the pattern.

    For instance, in Christopher Alexander's work, bus stops and waiting rooms in a surgery center are both part of the context for the pattern "A PLACE TO WAIT."

    I’ve spent most of my career working on the Mac OS/hardware and Windows OS/hardware The use of patterns seemed like a natural thing to do, but not intuitive for the people who deploy IT infrastructure.  With Gartner’s Pattern-Based Strategy, my persuasion challenge is dramatically decreased.

    So, what is good about Gartner’s Pattern-Based announcement?  Their first 2 paragraphs are well written to identify the need.

    Gartner Says Companies Must Implement a Pattern-Based Strategy™ to Increase Their Competitive Advantage

    Analysts Discuss the Framework for Implementing a Pattern-Based Strategy During Gartner Symposium/ITxpo, October 18-22, in Orlando

    STAMFORD, Conn., October 8, 2009 —

    The economic environment rapidly emerging from the recession will force business leaders to look at their opportunities for growth, competitive differentiation, and cost controls in a new way. A Pattern-Based Strategy will help leaders harness and drive change, rather than simply react to it, according to Gartner, Inc.

    A Pattern-Based Strategy provides a framework to proactively seek, model and adapt to leading indicators, often-termed "weak" signals that form patterns in the marketplace. Not only will leading organizations excel at identifying new patterns and exploiting them for competitive advantage, but their own innovation will create new patterns of change within the marketplace that will force others to react.

    They identify the need for closed loop feedback systems to measure the effectiveness of change.

    A CONTINUOUS CYCLE: SEEK, MODEL AND ADAPT

    Most business strategy approaches have long emphasized the need to seek better information and insights to inform strategic decisions and the need for scenario planning and robust organizational change management. Few have connected this activity directly to the execution of successful business outcomes. According to Gartner, successful organizations can achieve this by establishing the following disciplines and proactively using technology to enable each of these activities:

    For the same reason I added modeling and social networking to the list of things I discuss and blog about, Gartner explains.

    Modeling for pattern analysis — Once new patterns are detected or created, business and IT leaders must use collaborative processes, such as scenario planning, to discuss the potential significance, impact and timing of them on the organization's strategy and business operations. The purpose of modeling is to determine which patterns represent great potential or risk to the organization by qualifying and quantifying the impact.

    "Successful organizations will focus their pattern-seeking activities on areas that are most important to their organization," said Ms. Genovese. "Using models to do scenario planning will be critical to fact-based decisions and the transparency of the result."

    I have my black belt in Aikido, and one of the most important skills I figured out to be better is you must develop the skills to change.  Gartner adds this as well.

    Adapting to capture the benefits — Identifying a pattern of change and qualifying the potential impact are meaningless without the ability to adapt and execute to a successful business outcome. Business and IT leaders must adapt strategy, operations and their people's behaviors decisively to capture the benefits of new patterns with a consistent and repeatable response that is focused on results.

    Clients – I told you taking a modeling based approach to discover patterns with real-time monitoring systems will allow you to be ahead of the competition.  And, what better proof than Gartner now promoting the same ideas.  :-)

    Read more