“it looks like one of the key features not on the list of components for the California boxes is going to be a red discount pen”

Timothy references sources who have obtained a price list and shared it with “El Reg” . I wish Timothy would have contacted an actual Cisco Unified Computing System Advanced Technology Partner, because any partner that is involved in the launch could have explained to him the concepts of List price (List), Manufacturers Suggested Retail Price (MSRP), and Purchase or Buy price.

In this article I want to dispel the myths of server and network manufacturer pricing, demonstrate the true cost of building a data center with blade systems, and at the end provide a cost comparison between legacy server vendors options and Cisco’s Unified Compute System.

First, lets go over some the basic concepts of vendor pricing. At the end of this you should understand the difference between list price, manufacturers suggested retail price, and purchase price.

List Price

List price is a high level number that Cisco publishes weekly in its global price list. The purpose of this list price is to provide a uniform price list across all product sets that Cisco offers. The most important thing about list price is NOBODY EVER PAYS LIST PRICE. Let me repeat that again NOBODY EVER PAYS LIST PRICE. Are we clear? This is similar to list price on a car on the car lot. All list price provides is a starting point where a Cisco partner and a customer can negotiate a common discount and end up with something close to (generally at or below depending on technology type and yearly spend) MSRP.

Manufacturers Suggested Retail Price (MSRP)

This concept is something that anyone who has purchased a car before is familiar with. The number that is on the window of the car when you look on the lot is list price. The first number the dealer brings up lower then sticker is MSRP. Depending on the popular of the product, the competition in that particular space, and the negotiating power of the customer you will either pay that price, or some percentage below. For example if you are buying one new car you may have the negotiating power to get the price to drop 5% off of list. If you are buying 200 new cars (say a fleet) you have significantly higher negotiating power, and you may be able to drop the price by 15% of of list price.

In Networking Sales MSRP is significantly less then list price. A good exercise to see what this number is, is to find a device, say a WS-C3560E-12SD-E (3560 with 12 Gig SFP ports and 2 10 Gig ports) in the Global Price List. You have access to this at any partner level at www.cisco.com/dprg . (my point here is that this is no big secret). As of Friday June 12 2009 the LIST price for this product is $19,995.

Now take that same part number - WS-C3560E-12SD-E and pop it into your google search window. Within the top four links I found this product for $12,434.15 . This price is for  pure fulfillment, with no value added consulting or design work from you local Cisco partner.

If you do the quick math, this price difference is equal to 38% off of list price. Come to your own conclusions, but it would be safe to say that this could be considered MSRP for Cisco products.

Purchase / Buy Price

Buy price is just that, the price at which the customer purchases (buys) the product. This is can be at MSRP, or if the customer is buying significant amounts of hardware at a time, or if there is a “special” (programs and incentives) going on the number could be slightly lower then MSRP.

Percent off of list differences between legacy server vendors and networking vendors

This is where the biggest confusion is coming from. Legacy server manufacturers  have set their list prices much closer to MSRP then networking vendors (remember, MSRP is the price where most customers purchase at).

Why is this? In the networking space, vendors have historically created their own processors, ASICS and boards. This means that the sales discussions are feature to feature. It also meant that you had to have a conversation with the networking vendor or networking partner to properly size your network devices and get a quote – which is around MSRP, not List price.

In the legacy server space, especially the majority of the x86 server space, the market has been essentially commoditized. E.G. – You can buy an intel based server with X amount of memory and hard drives that will perform roughly equally from any of the main manufacturers. That made it much easier for a sever admin to just pull a price off of the web and compare. So what the server vendors ended up doing is setting their list price  only slightly above MSRP.

What this translates to is the list price, between legacy compute vendors and Cisco will be drastically unequal. What is equal is MSRP, or the generally accepted purchase price by common customers.

Why did Cisco set the list price of UCS higher then the legacy server manufacturers?

For the vast majority of its sales, Cisco relies on what is called the channel model. This means that Cisco partners with local Value Added Resellers (VAR’s) who sell Cisco’s products and then provide consultative services to design and implement them in customer networks. Most customers who purchase any regular amount of Cisco product either have a general expectation that they will buy Cisco product at a certain percentage discount off of list and sometimes the partner and customer have entered into purchasing contracts which require that all Cisco product is provided at a specific discount off of list price.

If Cisco decided to set the List price at a small percentage lift over MSRP, this would cause a problem for the entire channel. This would be especially hard for any customer who had a contract to buy product at a specific discount. What would happen is contracts would have to be renegotiated, which generally takes months and is about as fun as pulling teeth.

The second reason for setting list price the for compute the same as list for network is quoting. Right now, if you buy hundreds of different Cisco devices through a reseller it is very likely that the discount is going to be the same across all products. This makes the mechanics of sales much simpler, because you don’t have a lot of math in the quote (this can cause errors). On the customer side, having one set discount makes it much easier to compare quotes and to ensure that they are getting the best deal possible. In short, sticking with Cisco’s current list pricing structure benefits both the customer and the partner.

Now that we have set the record straight on list price, MSRP, and Buy price, lets take a deeper dive into what components make up a blade system powered data center. And then we will compare the price structures of both.

Components of all Blade Systems

Blade Server – The compute blade where commodity silicon elements such as the CPU and RAM are housed. As of writing this article, the latest high performance blades from all major server manufactures support two xeon 5500 processors (Nehalem) and DDR3 memory.

Mezzanine cards – These cards take the place of PCI-e cards in a rack form factor server. In a blade system these provide data network and storage network connectivity. They attach to the blade itself via proprietary connectors that implement either PCI-e 8 or 16 lane connectivity at the time of writing. In some cases other functions such as IO accelerators can also be attached in the mezzanine card form factor.

Blade Enclosure – This is functionally a tin can where eight to sixteen blades are placed. It also is used to provide a centralized power distribution fabric, as wells as slots for interconnections of data and storage network devices.

Data Network Modules – These are effectively ethernet switches that have been miniaturized to fit into the tight confines of a blade enclosure. Classically they have provided 1 gig connectivity to the servers, and 10 gig to the distribution layer, however with Nehalem processors and VMware there is a move towards presenting 10 gig connections to the server, and multiple 10 Gig connections into the distribution layer.

Storage Network Modules – The local disk in a blade server is classically anemic. To provide higher IOPS (input outputs per second) to disk, Fibre Channel connectivity is extended by taking SAN fabric switches and miniaturizing them to fit into the blade enclosure.

Data Network Distribution – If you have multiple blade enclosures there is a need to connect them together at a reasonably high bandwidth. To serve that need a variety of 10 Gig distribution switches are provided from all server manufactures at varying cost and performance levels.

Storage Network Distribution – Along the same lines of the data network distribution, SAN fabric switches have to aggregate up to a SAN distribution layer, or if the installation is reasonably large a “director” class SAN switch. This allows all the blade enclosures to see the same storage network, as well as providing for deterministic storage network performance as you scale out.

Management Infrastructure – All manufactures have a need to manage and monitor all of the devices that comprise their blade system. Many manufactures have multiple management modules per blade enclosure.

Comparison of Costs – Cisco vs Legacy Server Manufacturers

The funny thing, is that many people have assumed that Cisco’s Unified Computing System will be priced higher then legacy server manufactures products. In my mind this is because they associate higher quality with higher price (basically the Mercedes vs Kia discussion). Here is something that will shock you - it costs less to buy an entire blade system through Cisco then to buy from the legacy server manufacturers.

When people hear this, they are puzzled. How can two server manufacturers, who buy their CPU’s from the same company (Intel) and their memory from the same fabs end up with different prices? The answer is elegance in engineering. Lets go through each of the elements of a blade system infrastructure and find out where the costs are. More importantly lets look at where Cisco has innovated to provide higher performance at a lower cost.

Dollars and Cents – How much is the cost difference

I worked up two quotes recently. These quotes included all elements required to build an end to end blade system using both legacy server manufactures devices, and using Cisco’s Unified Computing System. I have broken out two scenarios.

8 blade servers - Cisco wins with a savings of 11%

In this scenario the cost of servers and enclosures were fairly equal. The cost savings started racking up as storage and data networking devices were included, as well as base management software was taken into consideration.

320 blade servers - Cisco wins with a savings of 31%

With 32o blade servers the same cost savings seen in the 8 server scenario were amplified. Economies of scale translated into significantly less devices being required to support the individual compute blades. This resulted in 31% savings compared to the legacy server manufacturers.

Summing it up

Cisco has entered into a highly competitive server market by taking an elegant approach to its blade systems. This approach lowers the purchase price of the UCS through reducing the amount of components compared to legacy server manufacturers. I know that there is a lot of misinformation flying around, and I hope this helps to set the record straight on the pricing of Cisco’s Unified Computing System.Similar Posts:

• Cisco Nexus 4000 Blade Switch
• Simplifying your Data Center with Cisco’s Nexus 2000 Fabric Extender (FEX)
• Cisco introduces the C-Series Rack Servers
• Nexus 5020 – Consolidated 10 Gig Ethernet and 4 Gig Fibre Channel
• Cisco’s Unified Computing System – It’s not just a blade center
• Is your network ready for Cloud Computing with Virtual Infrastructure 4?

--Colin McNamara

Confusion about Cisco UCS pricing – Setting the Record Straight

In that spirit, my schedule is listed below. If you are in the area, it would be great if you would stop by and say hello.
“6/23/08″ “11:00 AM”"Certification Exam  –  Certification Exam”
“6/23/08″ “1:30 PM”"BRKCCT-1001  –  Contact Center Welcome Session: Focusing on the Experience”
“6/23/08″ “5:00 PM”"WoS Reception 1  –  Welcome Reception in World of Solutions”
“6/24/08″ “8:00 AM”"certification focus group  –  certification focus group”
“6/24/08″ “9:00 AM”"BRKITI-1031  –  Cisco Data Center 3.0 Strategy and Business Impact”
“6/24/08″ “10:00 AM”"GENKEY-1001  –  Keynote and Welcome Address with John Chambers”
“6/24/08″ “12:00 PM”"ITIPCS-1015  –  NetQoS: Getting the Most from Cisco WAN / Application Acceleration Technologies”
“6/24/08″ “1:00 PM”"BRKDEV-1221  –  Applying Cisco’s Nexus Operating System (NX-OS) and DCNM APIs to Emerging Data Center Infrastructure”
“6/24/08″ “2:00 PM”"BRKDEV-1001  –  Cisco Application eXtension Platform”
“6/24/08″ “3:00 PM”"GENSSN-1001  –  Super Session: The Power of Collaboration Panel”
“6/24/08″ “4:00 PM”"BRKSEC-3007  –  Solving Security Challenges with Embedded Event Manager”
“6/24/08″ “7:00 PM”"BRKAGG-2001  –  Multiservice Edge Architectures and Solutions for Service Providers”
“6/25/08″ “6:00 PM”"CCIE Appriciation  –  CCIE Appreciation part – Nascar Grill”
“6/25/08″ “9:00 AM”"BRKDEV-1111  –  Location Based Services using Cisco Location API”
“6/25/08″ “10:00 AM”"GENKEY-1002  –  Cisco Technology Keynote with Padmasree Warrior”
“6/25/08″ “12:00 PM”"ITIPCS-1019  –  Fluke Networks: General Parts Uses Embedded IOS Technologies to Successfully Manage Inventory at Retail Locations:
“6/25/08″ “1:00 PM”"BRKDEV-1051  –  ANA Technical Session and Demo”
“6/25/08″ “3:00 PM”"GENSSN-1002  –  Super Session: The Data Center–Evolution and Transformation:”
“6/25/08″ “4:00 PM”"BRKDEV-1131  –  Customer Voice Portal Application Development”
“6/25/08″ “8:00 PM”"Customer Event  –  Customer Appreciation Event”
“6/26/08″ “9:00 AM”"BRKITI-1034  –  Realize Business Goals through Network Architecture Solutions”
“6/26/08″ “10:00 AM”"GENKEY-1003  –  Closing Keynote Address and Guest Speaker, Ben Stein, Actor/Writer/Columnist”
“6/26/08″ “1:00 PM”"BRKCCIE-3003  –  CCDE: The Cisco Certified Design Expert”
“6/26/08″ “3:00 PM”"BRKDEV-1171  –  Managing  Network Performance using the New IOS Data Collection Services”
“6/26/08″ “4:00 PM”"BRKDEV-1181  –  Configuration and Provisioning using IOS  XML API”Similar Posts:

• Cisco Live 2009 – Networkers class schedule
• Cisco Live 2010 Schedule
• Cisco Certified Architect – Board examination above the CCIE and CCDE
• It’s on like Donkey Kong – CCDE practical registration is open
• Where is Colin ? Passing the VCP exam (VMware Certified Professional)
• About Colin McNamara

--Colin McNamara

I’ll be at Cisco Live 2008 (networkers) in Orlando all week

When I work with data center infrastructure I expect the following – clean, remotely manageable, secure devices that runs on the same power and similar cabling, and everything can have a 24×7x4 support contract for hardware replacement. For the most part, you get this when dealing with Cisco, HP, Sun and similar manufacturers.

More often then not (with a few very cool exceptions), when I run into video surveillance infrastructure the video management infrastructure runs on some random third tier manufactured server. It never fails that the video management software is on Windows (normally XP or win2k). I have even seen some systems where the vendor requires you to have a session open to run the software.

And then when you get to the encoders themselves, it never fails. You have two choices.

1. The Uber package that can run a Casino, Identify and track dust mites , and if you point it at space, determine if there is life on mars.
2. Individual dinky encoders that run one or two camera’s each. They have limited encoding choices, limited camera control, no remote management, and normally run on 110 volt system that require different power distribution then the 220 that is common in systems today.

Cisco’s answer to this mess

Cisco has released both a video management solution, as well as a video encoding solution in a network module form factor for the Integrated Services Router (ISR).

The first part of this system, the Video Management and Storage System (VMSS) module fills the following roles -

• Management of multiple video streams from one interface, including IP cameras, 3rd party encoders, and streams from Cisco’s video encoding module
• Streaming of live and archived footage through a web browser interface
• This one is pretty cool – The module can mount external storage via iSCSI. So, in addition to its 160 gig internal drive, you can mount a filer and utilize external storage to scale the system.
• “fast forward” to events, as well as notify security and other personnel through SMS and email

The second part of the system (the module on the left in the picture above) is the Analog Video Gateway Network Module (EV-IPVS-16A). It has a couple functions -

• It can take up to 16 analogue video inputs and encode them with MJPEG or MPEG4 codecs
• You can use the first two ports to output video to a external monitors
• If you are using MPEG4, it can be used as a motion detector (handy for fast forwarding to important events, or triggering alerts)
• It can control pan and tilt cameras. This is good for pointing the camera at the janitor unplugging your servers each night to vacuum
• You can configure analogue contacts as an alarm. This can be bound to a door switch, or even temperature and water level monitors in a remote data center. This one will be very handy.

The third part of this solution is Cisco’s Video Surveillance Operations Manager. It manages, archives, displays and distributes the content that was created and collected on the two previous modules. You would use this if you had many branches to aggregate, or needed to staff a video wall (e.g. casino gaming commission operations). Now, you can run each of these components individually. Buy run together as a whole, Cisco has an enterprise class security solution.

Want to learn more ?

Branch office security page on cisco.com http://www.cisco.com/en/US/products/ps9671/prod_module_series_home.html

Cisco’s product page for the Video Managment Module – http://www.cisco.com/en/US/prod/collateral/modules/ps9671/data_sheet_c78_462225.htmlSimilar Posts:

• Interesting TechWise TV episode on virtualization
• Measuring and mitigating risk involved with sharing virtual infrastructure between DMZ and Internal environments
• Cisco NX-OS 4.0 | Next Generation Internet Operating System
• Nexus 5020 – Consolidated 10 Gig Ethernet and 4 Gig Fibre Channel
• Cisco Nexus 5020 and 5010 FCOE video ordering guide
• About Colin McNamara

--Colin McNamara

Simplifying remote site security with Cisco’s new video surveillance modules on the ISR

In the past couple years, VMware has changed from a product hidden in development and testing environments to a full fledged enterprise computing platform. It brings many benefits to the companies that implement it, however with those benefits come changes to the access layer of your data center. Your access layer is no longer a top of rack Cisco switch, or end of row aggregation chassis. It is now a virtual bridge that exists logically within your VMware ESX server.

This causes an interesting question to come up in many customers – Who is responsible for the configuration and maintenance of this Vswitch? At first glance most groups reference the port on the last Cisco switch as the division of responsibility between network operations and systems operations. This has worked well in the past for a three main reasons.

First, it divided responsibilities based on technical skillset. For example a network engineer understands spanning tree, trunking, routing protocols, firewalling. While a systems engineer understands file systems, databases and Linux and Windows operating systems.

Second, it provided for a interconnection point where standardized configurations could be applied by an operational group, versus complicated configurations that could impact overall network designs and require an architectural board review.

Third it provided for a clean hand off for troubleshooting. Both network and systems operations could agree on layer 2-4 functionality in an area that provided for detailed debugging on both sides.

Lack of a defined access layer

VMware ESX throws a wrench in this model. We no longer have this well defined edge at the access layer. The access layer now exists virtually inside a server. More specifically, it is a logical devices running in a Linux server. This presents a challenge because it requires cross over knowledge. Whoever is responsible for this integration has to be fluent in Linux systems administration , and also fluent in network design and operations. Frankly this is a rare skill set to come across, as it requires and engineer who has attained high proficiency in both systems and network engineering.

I see this fuzzy line of demarcation often as a failing point for many VMware integrations. Many times I see network operations teams not involved in ESX cluster design because its a “server” , and systems operations teams generally don’t have the networking skills necessary to design and implement an fully functional system.. The solution to this problem is education and collaboration.

The need for collaborative design sessions

The single most powerful element in a successful VMware integration is the creation of strong design documents. These are created by holding planning sessions where both your systems and networking leads hash out a strong design that takes both short and long term virtualization and network goals into account. Also, many times when people hear the word design, they think it is a high level Visio and a bill of materials. That is a just a fraction of the effort required. A proper design should cover everything from a 10,000 foot overview Visio down to protocol flow diagrams and configuration examples. By created a detailed design like this it is likely to bring up common issues such as 10 gig aggregation, trunking, VMotion security, layer two adjacency and layer 7 network service delivery on a white board instead of a production environment.

To create this detailed design, both your Network and Systems leads have to understand this product. VMware recognizes this is critical to successful implementation (and to further sales of their product) an offers the VMware Certified Professional certification. If you have the resources, I would recommend sending both your network and systems leads to this training at the same time. Having them attend training together allows them to leverage each others strengths and bring up questions specific to their network and their goals.

A real world example of this is the company I work for, Eplus. Last April forty of us, all senior engineers attended VMware Certified Professional training at the same time. The class was mixed up so there was an even distribution of CCIE’s, Systems Experts, and Storage Experts. Needless to say this presented our instructors with some extremely challenging questions, but more importantly it set the stage and created a venue for collaboration between these different practices within our own company.

Real world benefits

A great example of this model’s success this occurred last month. Rick and I were sitting in the engineering side of our Sunnyvale office, catching up on email after giving presentations at Cisco that morning and afternoon. In the bullpen behind us, one of the Microsoft architects was engrossed in a troubleshooting call with a large customer on the other line. It turns out a large systems vendor (who shall remain nameless) had been trying for a week to integrate the first ESX cluster into this network and just could not get the networking portion to work correctly. Our account manager received the call from a the customer, and asked the technical teams to step in to see if we could help out in any way.

The systems engineers were able to isolate the problem down to the network interconnections, but needed to bring in networking resources to resolve the problem. Rick and I were waved over and were given an overview of the problem and introduced us to the customer the far side of the call. We asked a few questions about the physical and logical architecture of their network and created a diagram of their network on the whiteboard. With this we were able to ask them to execute commands continuously isolating the problem domain until we found and resolved the issue.

Seven minutes had passed from the point Rick and I were waved over to the point the customer had a working installation. This allowed the customer to focus on moving their business forward instead of fixing a failed implementation. Three of us on the call had attended VMware Certified Professional training together. We had spent at a minimum 50 hours each creating a baseline of understanding in class, as well as many discussions in engineering meetings. The solution came in seven minutes not because of any one teams individual strengths, but because of collaboration. The systems engineers were able to isolate the problem domain very specifically. And as network engineers trained on VMware were able to quickly understand and digest the issues, and tie it together with our larger understanding of networks as a whole. Only at that point, when the team was able to leverage each others strengths were we able to address the problem so quickly.

There will come a point in the next few years where this fuzzy boundary between the “network” and the “server” is established again. My call is that this will coincide with Cisco finishing development of their Vswitch that will reside inside the ESX server. This switch will require both Cisco and VMware improve their design and integration guides for ESX which are both frankly lacking substance. Until those detailed architecture, integration and troubleshooting guides exist the key to successful ESX cluster implementation will be a strong cross trained systems and network teams that are collaborating on the next level of virtual network design in your enterprise.

Want to learn more?

Cisco – Integrating Virtual Machines Into Cisco Data Center Architecture

This is Cisco’s main design guide regarding the integration of virtual machines. You can use it as a decent high level overview if you are a network engineer who is curious how VMware ESX, or Xen servers for that matter will fit into your network.

VMware – Virtual networking Concepts

This VMware document goes between high level overviews and detailed descriptions. It is a decent resource for a network engineer, and provides an overview of ESX network features, however it misses the target for providing configuration examples.

Blog of Scott Lowe – Technical Lead for Virtualization at Eplus Technology

Scott is an engineer that works with me at Eplus Technology. He is based out of the east coast and covers servers, storage and virtualization. His blog is chock full of good of information. A recent post of interest was how to enable Cisco Discovery Protocol (CDP) on VMware ESX server network interface cards.Similar Posts:

• Cisco releases Nexus 1000V virtual switch for VMware
• Cisco Certified Design Expert – CCDE – officially released by Cisco
• Arista Networks – Their approach to cloud networking
• Resume – Colin McNamara, CCIE #18233
• Cisco NX-OS 4.0 | Next Generation Internet Operating System
• Cisco EMC and VMware partneship VCE VBlocks Acadia and the Partner Ecosystem

--Colin McNamara

Challenges integrating VMware into Cisco networks

IOS-XE is takes the best elements out of Internet Operating System (IOS) which has its roots in a closet at Stanford, and combines them with the most successful open source technology ever – Linux. Cisco is leveraging Linux virtualization technologies such as Kernel Based Virtual Machines to protect against operating system failures as well as to allow for In Service Software Upgrades (ISSU).

To really appreciate this, we first have to dive down into the overall architectural changes of the ASR1000. The largest change that Cisco has made was to implement separate forwarding and control planes. In the past, Cisco routers would have the processes responsible for forwarding traffic, and the processes responsible for configuring the router running on the same root operating system. The side effect of this is that if you want to upgrade the root operating system of your router, you are going to have interrupt the traffic flowing through it to do so, or have a physically separate route processor to take over while you rebooted. This is a big headache operationally, and effectively forced engineers to design in separate physical chassis to meet high uptime requirements.

What Cisco has done to address this, was to mirror changes made in their storage and carrier routing portfolios. Both of those product lines utilize the operating system to push commands into advanced processors that exist on the line cards themselves. The ASICS on the line cards are designed to work in a distributed fashion, so that production traffic never goes into up into the router processor (or sup engine). This in effect ensures that the control and forwarding planes can exist as independent elements.

If you look at the graphic above, you will notice 3 main zones. The upper zone is what we would normally describe as the control plane. This is where the higher level functions such as your routing processes, ssh daemons, snmp daemons, and shells live. In short, if you you configure or read something, you are going to do it here. The only time traffic flows through this plane is when you are doing a thing called process switching. keep in mind this is a rare occurrence and usually occurs because of an oversight in your network designs.

By separating the control and forwarding planes, this allows Cisco to basically run a management station on the router, that programs chip sets in the line cards on the fly. This in my opinion is where the true power of this architecture comes through. By separating the two functions the software engineers are free to utilize powerful open source technologies such as Kernel-based Virtual Machines, and the Linux kernel, while letting the integrated circuit engineers design blazing fast chips which allow full functionality at line rate.

What benefits should we receive from a virtualized control plane? First, in larger routing and switching chassis (including the top end of the ASR1000 line) you normally have physically redundant route processors (RP)/ supervisory engines(SUP). The operating systems on these RP’s synchronize many things, including configuration, process state, routing tables, security associations and much more. The primary reason for this, is if you have a failure in the active RP, you can failover to the standby RP without interrupting traffic flows.They also can be used to streamline the software upgrade process by only upgrading one RP at a time, and then gracefully transferring traffic to it. Once proper operation is verified, the backup RP can be brought up to the same code revision.In any production environment this is highly desirable, and helps immensely in the battle for five nines.

The ASR1000 takes the redundant RP concept seen in high end chassis, and allows you to implement redundant upgrades, as well as protection against software failure, with only one physical route processor. This is done by utilizing Linux kernel virtualization. Instead of running the control plane directly on the production hardware, a small kernel is inserted. Booting from that are two copies of IOS-XE. These run independently, and synchronize state and configurations just as if you had two physically separate route processors. What this means in operational English, is that where in the past, you would have to either have two devices, or a larger device with redundant RP’s to upgrade without disruption, you can now have that same ease of maintenance, in a much smaller (and at the end of the day, less total cost) package.

Below this is the forwarding plane.It plugs into to a high speed interconnected fabric which all line cards and RP’s are redundantly connected to. In the diagram above, this is the bottom level. Items in this plane include buffer memory, Cisco Express Forwarding (CEF) ASICS, and now the new QuantumFlow processor. This is normally where you would find your DCEF enabled line cards, fibre channel and Nexus7000 line cards, as well as the modules for the ASR1000 routers. When properly utilized, traffic should be relatively isolated to this tier, and function independently from the control plane.

The shining star of the ASR1000’s forwarding plane is a group of chips that is referred to as QuantumFlow. The QuantumFlow architecture itself merges Cisco’s strength in integrated circuit design, with its strengths in IOS software design. In the past, Cisco would design ASICS’s for specific functions, and then write commands down into them. This has worked very well, until they point that a new feature came out that couldn’t leverage the fixed configuration of an older ASIC. Your choice at that point was generally to process switch for that feature (which is slower, and honestly bad form), or upgrade your cards to the newer ASIC design. The QuantumFlow chipset approaches this problem from a new angle. The first chip in the set (Popeye) is designed to be field programmable in C, as well as no fixed internal pipelines. This combined with utilizing 40 cores running between 900 and 1200 megahertz allows the programmers to utilize parallel processing techniques to utilize an immense amount of processing power in real time.

To put things into perspective, remember when you got your first multi core laptop or desktop. You were able to say watch a DVD, as well as compile code at this same time, while continuing to have a responsive workstation. Now imagine what you could do with a 40 core processor. This is the kind of power that we are talking about. Now imagine, that not only is your workstation immensely powerful, but you could also offload common jobs such as running daily builds, or encoding videos to another machine (or in this case processor.

In the ASR1000 this processor is called Spinach (yellow are in the graphic above). And of course just like the cartoon, Popeye’s potential really comes to light when combined with Spinach. Spinach is a separate chip, that is used a a traffic manager. This chip handles queueing and quality of service, ensuring that the proper packets arrive at the proper time, as well as interconnecting with cryptographic offload engines so it can equally apply services to encrypted flows.

At the end of the day, the most important question is not how fast something is, or how cool it is. The question is what can it do for me? By leveraging this new architecture the ASR1000 is now able to do line rate inspection of traffic using Network Based Application Recognition (NBAR), Support 128,000 queues for deep quality of service, secure and encrypt data using zone based firewalls and embedded crypto engines, segregate traffic using MPLS, integrate advanced voice and video functionality, as well as providing fulling Netflow v9 support for all of the above. It provides all of these services in an always on solution utilizing Linux virtualization, as well as leveraging an flexible chip set architecture that allows for field programmable improvements in the future.

My hope is that after reading this article that you are in a better to understand how Cisco is leveraging open source technology and integrated circuit designs to improve the foundation of the internet. In upcoming articles I will be discussing design scenarios utilizing this features in this product, as well as highlighting other areas where Cisco is embracing both open source technology, as well as open architectures that can properly leverage projects such as Linux, Ntop, Wireshark and more. If this article has you interested in learning more about some of the technologies mentioned today, then I encourage you to check out some of the links below, or shoot me and email to be highlighted in a future readers questions article.

Learn more about Linux Kernel-based Virtual Machines

Learn more about Cisco’s ASR1000

Learn more about Cisco QuantumFlowSimilar Posts:

• Cisco NX-OS 4.0 | Next Generation Internet Operating System
• Application Extension API notes – Cisco Live 2008
• Arista Networks – Their approach to cloud networking
• Zone based IOS firewalls
• Simplifying remote site security with Cisco’s new video surveillance modules on the ISR
• Altor Virtual Network Security Analyzer (VNSA) integrated with Cisco’s Nexus 1000v for VMware

--Colin McNamara

Cisco is using Linux virtualization and 40 core CPU’s for its next generation routers

I was wrong, the Green Data Center is not about saving baby seals, it is about saving cold hard cash. Saving the world is just a nice side benefit.

That being said, saving cold hard cash is a very important discussion item in any IT Operations group as they are normally seen as a cost center. For them, a penny saved is literally a penny earned. Not only can you save money by not paying for power, but PG&E will actually has a budget to pay you NOT to use their power. Most people here this and get a puzzled look on their face. “why would the power company, who makes money on power, not want me to buy it from them?” The answer is that Californians use more power then PG&E can produce at peak times. When they have to buy it from another state it can cost them 10 times or more then they charge us. This is the reason why PG&E will pay you to use less. Each penny they give to the consumer for saving a watt, saves them 4 pennies (80% return on investment).

Great, PG&E saves money by giving it to me. How do I get this cash? Well there are a couple ways to get this.

1. Incentives for new buying new energy efficient servers
2. Rebates for moving to virtualized servers
3. Rebates and incentives for moving to thin client desktop systems
4. Audit teams for cooling and power if your Data Center is 10,000 square feet or more
5. Incentives for airflow control systems
6. Incentives for high efficiency UPS and power distribution systems
7. Technical services for cooling system evaluation (PG&E funded)

That is a pretty comprehensive list of how to get money from the power company, but you can save even more money buy not using the power in the first place. Not unsurprisingly this starts with the server.

First thing you can do, is virtualize, virtualize, and virtualize some more. For most people this means VMware. For others this may mean Xen, or Microsofts virtualization product. Whatever flavor you chose, the key message is to consolidate from many servers to few. A server sitting “idle” still pulls 50% of its max current. Now, howe many servers do you have that are just sitting there? My guess is a large amount. By virtualizing these servers, you allow them to be stacked onto high performance server that can be run at a higher utilization. This lowers the over all power utilization for your DataCenter. Another side benefit is that ever watt that you remove from a server, you get another watt removed from your cooling.

These same virtualization techniques can also be applied to your network devices, which account for 6 to 12 percent of your datacenters power draw.

Ask yourself a few questions

• ” Do I need 4 different firewall clusters?”. It is likely that these are leftovers from organic growth, and that you could consolidate them into virtual firewalls on a more efficient chassis (ASA comes to mind).
• ” Do I need to maintain physically separate infrastructure?”. There are technologies like MPLS, VFR-Lite, Virtual Switching and more that allow you to consolidate onto a shared network infrastructure, taking a service provider approach to providing transport in your network.
• ” Am I running old inefficient gear?”. Power supplies have increased in efficiency over the last few years. There may be a good return on investment for you to upgrade.
• ” Can I consolidate into larger chassis?”. Ask the question, which is more efficient – a closet full of 3560’s or a 4507? There is efficiency in scaling out.

I hope that reading this has caused you to ask some questions, and maybe look at the larger impact of your network operations on both the ecosystem and your operational expenses. With these questions in hand, you might want to talk to PG&E and your Cisco / HP parter about going “Green” in the data center.Similar Posts:

• Is your network ready for Cloud Computing with Virtual Infrastructure 4?
• Usability features in Cisco’s Nexus 7000
• Cisco’s Cloud Computing Offering
• Cisco introduces the C-Series Rack Servers
• New features in VMware 3.1
• Simplifying remote site security with Cisco’s new video surveillance modules on the ISR

--Colin McNamara

Moving towards a Green Data Center – Truth behind the hype

What useability enhancements do you feel are the most beneficial?

1. A separate, IP enabled, Management Interface. This has been a long time coming. The out of band management interface is very similar to a Ilo card in the HP world. it is effectively a supercharged console server that happens to site on the backplane of the sup engine. I am sure whoever pushed this feature through is going to get flowers one day from a Tech who DIDN’T lock himself out because the management interface was effectively a separate system.
2. Finally, a functionally USB Interface that I can transfer IOS (well, now NX-OS) images through. Everyone has a USB key nowadays, even my Grandmother has one, it will make life so much easier when I can have a 4 gig key with me that has most IOS / NX-OS  versions and my common configs and just pop them right in.
3. The integrated Cabling system is CLEAN. I love that it forces you to reserve the appropriate space for cabling, and that there finally is the possibility to avoid the flying spaghetti train wreck we see so often in Data Centers.
4. Front to back Cooling. The cooling design is well thought out. I liked the fact that it draws from directly above the front floor and exits rear top.. This should help out in raised floor data centers that have a large temperature gradient as you move to the top of the rack. It also negates problem of having multiple 6500 chassis side to side and having warm air blowing from the exhaust of one 6500 to the intake of another 6500.
5. Fan Slots are now placed where it is IMPOSSIBLE to cover with cables. I would say 7 out of 10 times when I walk into a new customers Data Center I find that there are cables run directly over the fan tray with no slack. That is not a failure in design per say, but it could have been avoided. With the Nexus 7000 fan trays in the back the problem is solved before it is created.
6. Power supplies are in the back . FAR away from the data cabling. It never fails that 20 amp circuits get uncomfortably close to copper cabling. By moving the power supplies to the back side of the chassis, this becomes a mute point and we remove any shadow of a doubt about EM interference causing craziness in our cabling.
7. This one sounds really mundane, but a quick heads up grouping of status lights. In the past these were normally in a position where you had to squat down to see them, or they are obscured by cables. Buy putting them on the front of the cable tray assembly it ensures these will always be visible.

What can we focus on now to make it a better platform?

1. One thing that worried me a little was the placement of the compact flash cards in the supervisory module. For those how haven’t it up close look at this picture of the chassis  and look for the Grey cover midway up the sup modules in the center slots. Behind them are two flash cards, one for system partition extension, and one to dump log files into. Having these cards available are great features however I could see an operational process of security rotating out the log partitions, or more likely and engineer pulling the flash card after dumping some data for analysis to it, and then pulling the wrong card by accident. Having a simple strap (like the screw downs for power supply plugs) or something similar would go along way towards mitigating that risk.
2. Continue with the spirit of innovation that has defined Cisco over the years. Cisco has consistently came out with or acquired and integrated many great products that directly address the needs of the market place into the product line (MARS, ASA, AireSpace, TelePresence, MDS, ACE, Etc) but frankly the last GAME CHANGING product that set the industry on its heals and forced everyone to rethink how we utilize technology to accelerate business as a whole was the acquisition of Selsius and the introduction of VOIP as an enterprise class product to the world. I remember having the hair stand up on my arms from the excitement of going up against Avaya and Nortel back then and fighting that uphill battle, educating customers and peers about this “new thing called VOIP and how CallManager (now Unified Communications Manager) is your ticket towards productivity.

   When we talk about the Virtual DataCenter, I/O Virtualization (FCOE) and VFrame Automation it is not just another incremental improvement of existing technology. It is a paradigm shift, a leap ahead, a GAME CHANGER. I get the same chills that I did when VOIP was new because I know that those are technologies that will force us to rethink how we approach computing and data systems. These technologies are to the Data Center what IP telephony was to the PBX, and Cisco is the only company with technologies and engineering know how in all the verticals necessary to pull this off.

Similar Posts:

• Simplifying your Data Center with Cisco’s Nexus 2000 Fabric Extender (FEX)
• Cisco Nexus 7000 DataCenter switch released – Welcome to DataCenter 3.0
• Nexus 5020 – Consolidated 10 Gig Ethernet and 4 Gig Fibre Channel
• Simplifying remote site security with Cisco’s new video surveillance modules on the ISR
• Moving towards a Green Data Center – Truth behind the hype
• New features in VMware 3.1

--Colin McNamara

Usability features in Cisco’s Nexus 7000

One feature that is new, and frankly extremely exciting is Virtual Device Contexts. Each virtual device runs with its own process, vs the use of tagged differentiators in technologies such as VRF-Lite. This provides for paravirtualized management instances, and clear lines of delineation for both software and hardware for a resource that can be shared between different groups within an enterprise.

Chassis that run NX-OS will support In Service Software Upgrades (NSSU) to allow operations groups to upgrade operating systems with zero downtime. This is accomplished through a combination of modular software architecture, and the decoupling for the control and forwarding planes.

One of my favorite features in SAN-OS is the embedded is fabric analyser. This is a tool that can sniff management traffic without having to plug in a sniffer, or provision a span port. You can dump in real time to a tcpdump like interface in the command line, output to a local file, or map to the ip of a wireshark instance that layer 3 access to the management port. Cisco again has taken the best of SAN-OS and bundled it with NX-OS. You will be able to remotely span management traffic without having to set up rspan, or trudge down to the datacenter to set up a sniffer.

Now, your router can call home right now so that is not a totally new feature. Smart Call Home was released recently into IOS. But that still doesn’t stop it from being a great feature. This allows you to configure NX-OS powered devices to mail an xml formatted troubleshooting email to TAC, and / or your support staff. This has been proven to drop the average time to resolution from 16-30 hours to 6 hours.

Now the drum roll…… All IP routing features are VRF aware. This has been a point of contention with me for a while. As Cisco and the market in general has embraced virtualization as an answer to pressing business concerns of leveraging shared infrastructure, while retaining security controls segregating disparate environments technologies such as MPLS and VRF within the datacenter have become more and more prevalent. That is great, however it never fails that the feature you need at that moment always seems to be coming out in the NEXT IOS release. With Cisco NX-OS 4.0 this is no longer a question.

Now, if I was a CIO and I was reading about all these new technologies that Cisco was pushing with NX-OS, I would frankly be cautious, and rightfully so. The thing is, most of these features are not new, they have been in use, and in production under the most stringent uptime conditions in the world – storage networking. They have been tried and tested on Cisco’s MDS line of storage networking switches. So get comfortable, get educated, but most importantly get on board for DataCenter 3.0.Similar Posts:

• Cisco is using Linux virtualization and 40 core CPU’s for its next generation routers
• The emergence of MDS features in Cisco’s datacenter networking equipment
• New features in VMware 3.1
• Link Round Up – L2TPv3 FCOE Trill Wounded Warriors
• About Colin McNamara
• Arista Networks – Their approach to cloud networking

--Colin McNamara

Cisco NX-OS 4.0 | Next Generation Internet Operating System

* Solid State Drive (SSD) boot support
As initially discovered last month, VMware will make available a special version of ESX Server (mentioned with terms like ESX Lite and Embedded ESX) for OEM vendors, to be installed into bootable Solid State storage devices (flash drives, etc.). This option will allow creation of ESX Server hardware appliances for easy jumpstart, granting smaller form-factors and improved reliability.
Dell, IBM and possibly other vendors will offer this option at announcement time in Q3 2007.
* DMotion
Unofficially introduced with ESX Server 3.0.1, in its first version DMotion is a special VMotion operation only capable of moving running virtual machines from an ESX Server 2.5.x host to a new ESX Server 3.x., without shared SAN LUN mandatory requirement.
In ESX Server 3.1 this capability will be extended, allowing hot migration of running virtual machines between ESX 3.1 hosts through the Ethernet cable.
* Patch management system for host and virtual machines (Update Manager 1.0)
ESX Server 3.1 will finally introduce an automated patch management system called Update Manager. This solution will be able to update both host itself and virtual machines (both Microsoft Windows and Red Hat Enterprise Linux).
Update Manager will look for available updates from Shavlik Technologies website (a possible acquisition after IPO), and will allow VI administrators to decide which patches to deliver to virtual machines.
Before applying them, Update Manager will take a snapshot and will even rollback automatically if something goes wrong.

(this product was originally codenamed VM Integrity and its developement started more than one year ago, when virtualization.info discovered it in June 2006)
* VMware Consolidate Backup (VCB) and VMware Converter 4.0 integration
VirtualCenter 2.1 will now allow restoring VCB images with an integrated version of VMware Converter, which reaches 4.0 release number.
* Server consolidation advisor
VirtualCenter 2.1 will expose a server consolidation assistant able to analyze which physical machines should be converted in virtual ones, and where to move existing VMs among available hosts.
(note that with this feature VMware is further extending competition with PlateSpin, covering both features with PowerRecon and PowerConvert)
* Guest OS disaster recovery capability
VirtualCenter 2.1 will be able to recognize a failure inside a virtual machine and restart it through VMware HA module.
* Support for VMware Server 2.0
VirtualCenter 2.1 will be finally able to seamless manage both ESX Server and VMware Server 2.0 hosts.
* Lockdown Mode
ESX Server 3.1 will expose a new security feature to completely disable local administrative account after a VirtualCenter 2.1 takes remote control.
* Power saving capability (Distributed Power Management)
VirtualCenter 2.1 will introduce a new resources utilization analysis feature, able to verify when a physical host can be powered off, VMotion-ing its virtual machines on other hosts without impacting performances.
* Support for Cisco Discovery Protocol (CDP)
VirtualCenter 2.1 will be able to recognize and use CDP to discover physical and virtual network topologies.
It stays unconfirmed if ESX Server 3.1 will already expose new virtual network architecture, allowing 3rd party virtual switches, as it will be announced by Cisco CEO at VMworld 2007.
* Support for 10Gbit Ethernet network cards
* Support for TCP/IP Offload Engine (TOE) network cards
* Support for network load balancing algorithms
* Support for 200 hosts and 2000 virtual machines
* Support for 128GB RAM per host and for 64GB RAM per virtual machine
* Support for SATA storage devices
* Support for N_Port ID Virtualization (NPIV)
* Support for VCB over iSCSI SANs
* Support for IPv6 in virtual networking
* Support for Para-virtualization guest OSes

Copyright ©2008 | Colin McNamara | CCIE 18233 | All Rights Reserved”

Colin McNamara
CCIE #18233

http://www.2cups.com

“The difficult we do immediately, the impossible just takes a little longer.”Similar Posts:

• Cisco releases Nexus 1000V virtual switch for VMware
• Is your network ready for Cloud Computing with Virtual Infrastructure 4?
• Cisco EMC and VMware partneship VCE VBlocks Acadia and the Partner Ecosystem
• Cool new features in 12.2(33)SXH
• Cisco’s Unified Computing System – It’s not just a blade center
• Interesting TechWise TV episode on virtualization

--Colin McNamara

New features in VMware 3.1

Link – http://www.cisco.com/application/pdf/en/us/guest/products/ps8258/c1161/cdccont_0900aecd80679ce3.pdf

From Cisco’s site -

Cisco IOS Software Release 12.4T integrates a portfolio of new capabilities, including security, voice, and wireless, with powerful hardware support to deliver advanced services for enterprise and access customers. It will be issued as a series of regularly scheduled individual releases, which Cisco will ultimately consolidate to form the next major release.

Release 12.4(15)T, the sixth release of the 12.4T family, streamlines the Cisco IOS Software upgrade process, provides sub-second link failure detection and faster convergence, delivers next-generation Layer 2-7 flexible packet classification, enhances intrusion protection and SSL VPN capabilities, and provides support for the new Cisco 7201 Router, amongst other features.

Like all releases in the 12.4T family, Release 12.4(15)T integrates innovations that span multiple technology areas, including Cisco IOS Security, Voice, Cisco IOS Infrastructure, Access, High Availability, Management Instrumentation, Quality of Service, IP Multicast, Broadband, IP Routing, and IP Services. Release 12.4(15)T delivers these integrated technologies on the broadest range of hardware in the industry, including the Cisco Integrated Services Routers, Cisco 7200 Series, and Cisco 7301 Router.
Copyright ©2008 | Colin McNamara | CCIE 18233 | All Rights Reserved”

Colin McNamara
CCIE #18233

http://www.2cups.com

“The difficult we do immediately, the impossible just takes a little longer.”Similar Posts:

• The emergence of MDS features in Cisco’s datacenter networking equipment
• About Colin McNamara
• Application Extension API notes – Cisco Live 2008
• Simplifying remote site security with Cisco’s new video surveillance modules on the ISR
• New features in VMware 3.1
• Darrel Hinshaw – New Triple CCIE [Storage]!!!!!!!

--Colin McNamara

Cool new features in 12.4(15)T