First and foremost it is important to discuss how many networks are right now (click for larger images)

Figure 1.1

This is a pretty common setup, with 80% of so of systems still physically attached to a mix of 100 and 1000 Megabit access layer switches. The other 20% of virtual systems are attached as through blade center switches with 10 gig to distribution or larger (8-16 core) systems with bundled uplinks to the distribution switches. Service aggregation such as firewall, load balancing, and wan acceleration , image deployment, monitoring / management and other key Data Center services generally provisioned off 1000 Megabit ports in the distribution.

Last but not least, a shadow storage network runs connected to a small percentage of physical servers, and connected to all of the virtual servers via Fiber Channel, iSCSI, or NFS presentation. One thing to keep in mind that all of these elements may not be configured in the “optimal” SRND setup, but it is something that you can expect to see in a real life network today.

What’s so special about Vi4 Application vServices ?

Figure 1.2

The one new feature that is going to throw your network on its heels is vApp. Imagine if any application could be installed on any server in any location of your network at any time. What vApp enables you to do is create a portable application, similar to a Java application that installs in your web browser. But this application can be dynamically deployed to any virtual system in your Data Center as needed in response to a new application request, or the need to dynamically scale an application. What this means to us as network engineers is that any corner of our networks where virtualization is present can become a hot spot for critical application flows. This introduces a new dynamism to our fabrics which wasn’t there before, and frankly many networks are not equiped to handle it successfully.

Currently, to provide virtual machine redundancy we have VMware HA, where we both monitor the availability of a virtual machine. If there is a problem we can restart that virtual machine on another ESX host. With Application vServices there are many new elements and traffic flows. The two most important ones are vLockstop and vCenter Data Recovery. VMware is taking high availability to the next level by keeping a hot standby VM running on a second physical ESX server. If you think about it, you now are adding both additional latency sensitive heartbeat traffic as well as creating a situation where your storage traffic flows can be highly volatile. Additionally vData Center Recovery will be throwing traffic in new and interesting ways across your links.

Adjusting your network designs to deal with the cloud

Figure 1.3

First and foremost application virtualization needs a front end, in network engineering circles we have been handling this successfully for a long time with content switches (load balancers). These provide the logical rallying point for dynamic cloud applications. Since more and more systems will be utilizing these services it is important to ensure that your current content switches have headroom to grow, and if you don’t have any content switching capabilities, it is probably time to take a look at adding them to your data center.

Since applications can exist in any corner of the network, dynamic provisioning of storage and network connections has become critical. Maintaining “shadow” storage networks can provide some dynamic access to storage, however it is now becoming advantageous to virtualize your storage fabric along with your systems and network devices. Fibre Channel Over Ethernet (FCOE) provides just that.

If you look at figure 1.3 above, you will notice a new color introduced into the diagram, as well as the “shadow” storage network removed. This is possible because all the orange links run Data Center Ethernet (DCE) which provides a lossless path for FCOE to follow. The ESX servers now only connect into the Nexus 5000 switches. As you can see, we also have removed the shadow storage network, as it is now consolidated onto our new data center fabric. There may be use cases where we need to extend classic fibre channel connectivity out to certain hosts, and we can do that of the Nexus 5000. However if all possible it is advantageous to utlize FCOE to gain storage mobility and higher bandwidth for your hosts.

You may notice that our uplink counts have doubled. Since we are moving both storage and data traffic over the same links, as well as supporting vMotion and other bandwidth intensive network applications it is time to make the push to port channeled 10 gig adapters. Luckily prices have dropped considerably, where it is not cheaper to use 10 Gig then to bundle 8 1 Gig adapters together.

Last but not least you will notice the core switching is a different color. This is because the Nexus 7000 has found its home in the data center. I wont go to deep into the nexus as that is an article in and of itself. What I will say is that it is the best platform to use to aggregate the amount of 10 Gig links that are populating the data center in a highly available fashion. If you want to learn more about the 7000 I recommend reading these previous articles here here and here, as well as Cisco’s Data Center Switching page.

Your network once Virtual Infrastructure 4 (Vi4) and the Nexus 1000V are released

Figure 1.3

Fast forward to early summer 2008 VMware Virtual Infrastructure 4 (Vi4) and Cisco’s Nexus 1000V are released. Of course a new major version of VMware running your compute cloud, application vServices are in effect, vLockstep is running, and many other features that go with the platform such as the Nexus 1000V.

The Nexus 1000V brings a long missing feature to the Data Center, a defined network edge. Since VMware has taken hold in the Data Center, the boundary layer between the virtual machine and the network has devolved to a dumb bridge running in the memory of a ESX server. Installing the Nexus 1000V in your ESX servers creates a virtual switch with interface counters, pvlans, access controls, QOS and many other features that are critical to operating a Data Center. (check out a previous article about the Nexus 1000V).

Enjoying the fruits of our labors

Cloud computing in general, and specifically Virtual Infrastructure 4 have specific benefits that will drive efficiency and agility in IT as a whole. The mechanisms for these benefits will put increasing load on the storage and data networks in your Data Center. It is our responsibility as network architects to take a proactive stance and provision a network with the immediate future in mind. Luckily planning and preparing for these changes in advance have both benefits for our current infrastructure, as well as allowing us to enjoy the fruits of our labors as Cloud Computing changes from a buzz word to a reality.Similar Posts:

• Where is Colin ? Passing the VCP exam (VMware Certified Professional)
• Cisco releases Nexus 1000V virtual switch for VMware
• VMworld 2009 Schedule
• Altor Virtual Network Security Analyzer (VNSA) integrated with Cisco’s Nexus 1000v for VMware
• Cisco’s Cloud Computing Offering
• Measuring and mitigating risk involved with sharing virtual infrastructure between DMZ and Internal environments

--Colin McNamara

Is your network ready for Cloud Computing with Virtual Infrastructure 4?

1. Layer 2 Tunneling Protcol (V3) static and hairpin configuration example - my buddy Rick was nerding it out in the lab and sent a great configuration doc for L2TPv3 my way. L2TP(V3) is used to create a layer 2 psuedowire across layer 3 routed links. This is a great service provider tool that you can use in your own network, no MPLS needed .

2. SNIA Education – Fiber Channel Over Ethernet – There is a lot of buzz going around right now about Fiber Channel Over Ethernet (FCOE). There is also a lot of misunderstanding about the fundamentals of this architecture. This Storage Networking Industry Association (SNIA) does an outstanding job of covering FCOE at both at an architectural level, as well as going over low level messaging structures.

3. Trill (Rbridge) architecture – IETF internet draft – I think the last time I was this interested in an internet draft was when iSCSI was first being proposed in the IP Storage working group. Trill, in my opinion is basically a light weight version of MPLS / VPLS. It has as far as I can tell most of the advantages of this architecture, without some of the configuration and hardware requirement drawbacks. Fair warning, reading this document started a doc hunt that killed my Saturday.

4. Cisco’s Security Response to Sebastian Muniz’s IOS rootkit – Security is a very important aspect of network design. Sebastian’s IOS rootkit demonstration is going to force some customers who in the past have been “OK” with having older, possibly vulnerable IOS versions floating around to update their operational practices and start keeping their routers and switches operating systems as often as they do their servers. Thankfully, Cisco has been embracing technologies such as kernel virtual machines, in service software upgrades and more to lesson or remove the impacts of software upgrades.

5. Turning Wounded Warriors into Network Ninja’s – As a former Marine (well, always a Marine, formerly employed by the USMC) this program goes straight to the heart. Cisco is partnering with Naval Medical Center San Diego (NMCSD, or Balboa Naval Hospital for us locals) to provide technical training to Marines and Sailors who have recieved service ending wounds in Afghanastan and Iraq.Similar Posts:

• Fibre Channel over Ethernet is taking off
• Cisco NX-OS 4.0 | Next Generation Internet Operating System
• Zone based IOS firewalls
• Nexus 5020 – Consolidated 10 Gig Ethernet and 4 Gig Fibre Channel
• Cisco is using Linux virtualization and 40 core CPU’s for its next generation routers
• Cisco Nexus 7000 DataCenter switch released – Welcome to DataCenter 3.0

--Colin McNamara

Link Round Up – L2TPv3 FCOE Trill Wounded Warriors

What useability enhancements do you feel are the most beneficial?

1. A separate, IP enabled, Management Interface. This has been a long time coming. The out of band management interface is very similar to a Ilo card in the HP world. it is effectively a supercharged console server that happens to site on the backplane of the sup engine. I am sure whoever pushed this feature through is going to get flowers one day from a Tech who DIDN’T lock himself out because the management interface was effectively a separate system.
2. Finally, a functionally USB Interface that I can transfer IOS (well, now NX-OS) images through. Everyone has a USB key nowadays, even my Grandmother has one, it will make life so much easier when I can have a 4 gig key with me that has most IOS / NX-OS  versions and my common configs and just pop them right in.
3. The integrated Cabling system is CLEAN. I love that it forces you to reserve the appropriate space for cabling, and that there finally is the possibility to avoid the flying spaghetti train wreck we see so often in Data Centers.
4. Front to back Cooling. The cooling design is well thought out. I liked the fact that it draws from directly above the front floor and exits rear top.. This should help out in raised floor data centers that have a large temperature gradient as you move to the top of the rack. It also negates problem of having multiple 6500 chassis side to side and having warm air blowing from the exhaust of one 6500 to the intake of another 6500.
5. Fan Slots are now placed where it is IMPOSSIBLE to cover with cables. I would say 7 out of 10 times when I walk into a new customers Data Center I find that there are cables run directly over the fan tray with no slack. That is not a failure in design per say, but it could have been avoided. With the Nexus 7000 fan trays in the back the problem is solved before it is created.
6. Power supplies are in the back . FAR away from the data cabling. It never fails that 20 amp circuits get uncomfortably close to copper cabling. By moving the power supplies to the back side of the chassis, this becomes a mute point and we remove any shadow of a doubt about EM interference causing craziness in our cabling.
7. This one sounds really mundane, but a quick heads up grouping of status lights. In the past these were normally in a position where you had to squat down to see them, or they are obscured by cables. Buy putting them on the front of the cable tray assembly it ensures these will always be visible.

What can we focus on now to make it a better platform?

1. One thing that worried me a little was the placement of the compact flash cards in the supervisory module. For those how haven’t it up close look at this picture of the chassis  and look for the Grey cover midway up the sup modules in the center slots. Behind them are two flash cards, one for system partition extension, and one to dump log files into. Having these cards available are great features however I could see an operational process of security rotating out the log partitions, or more likely and engineer pulling the flash card after dumping some data for analysis to it, and then pulling the wrong card by accident. Having a simple strap (like the screw downs for power supply plugs) or something similar would go along way towards mitigating that risk.
2. Continue with the spirit of innovation that has defined Cisco over the years. Cisco has consistently came out with or acquired and integrated many great products that directly address the needs of the market place into the product line (MARS, ASA, AireSpace, TelePresence, MDS, ACE, Etc) but frankly the last GAME CHANGING product that set the industry on its heals and forced everyone to rethink how we utilize technology to accelerate business as a whole was the acquisition of Selsius and the introduction of VOIP as an enterprise class product to the world. I remember having the hair stand up on my arms from the excitement of going up against Avaya and Nortel back then and fighting that uphill battle, educating customers and peers about this “new thing called VOIP and how CallManager (now Unified Communications Manager) is your ticket towards productivity.

   When we talk about the Virtual DataCenter, I/O Virtualization (FCOE) and VFrame Automation it is not just another incremental improvement of existing technology. It is a paradigm shift, a leap ahead, a GAME CHANGER. I get the same chills that I did when VOIP was new because I know that those are technologies that will force us to rethink how we approach computing and data systems. These technologies are to the Data Center what IP telephony was to the PBX, and Cisco is the only company with technologies and engineering know how in all the verticals necessary to pull this off.

Similar Posts:

• Cisco Nexus 7000 DataCenter switch released – Welcome to DataCenter 3.0
• Simplifying your Data Center with Cisco’s Nexus 2000 Fabric Extender (FEX)
• Nexus 5020 – Consolidated 10 Gig Ethernet and 4 Gig Fibre Channel
• Simplifying remote site security with Cisco’s new video surveillance modules on the ISR
• Moving towards a Green Data Center – Truth behind the hype
• Interesting TechWise TV episode on virtualization

--Colin McNamara

Usability features in Cisco’s Nexus 7000

As with any vendor specific event, there is a mix of for public knowledge and for private consumption content, so I generally choose not to talk or write about subjects that may have been covered in the VT unless I can find some public documentation on that subject. So don’t expect to find any juicy pre-release information or gossip here. I don’t want to have the NDA police knocking at my door, and its just not cool to let stuff slip. So, I will generally avoid the subject.

What I can tell you is this – There is a lot of buzz about the Nexus 7000. It is a rocking platform, and we spent the majority of a day going over it. I can’t share much more then I did the night before the VT just yet (will wait till I get lab access to one) but I can share this.

Yes, most people are proud of their shots with Tom Cruise, or Oprah, Richard Stephens or BSD Girl. But I can Top that.. I have a picture of me and the Nexus 7000.

The DC Channels team was nice enough to take us down to the DataCenter and Network Applications (DNA) lab. Where the Nexus 7000 has taken its new throne. They allowed to ooh and ahh and poke and prod it. Weirdly enough, the one thing that struck most was the attention to detail that went into the physical design of this chassis. It is not only good looking, but has some super usability enhancements that really impressed me.Similar Posts:

• Darrel Hinshaw – New Triple CCIE [Storage]!!!!!!!
• Where was Colin in 2010?
• Cisco releases Nexus 1000V virtual switch for VMware
• Humor inside the Nexus 5000 switch fabric architecture
• BIG Cisco – VMware announcement – 1:30 Pacific time
• Measuring and mitigating risk involved with sharing virtual infrastructure between DMZ and Internal environments

--Colin McNamara

Me and the Nexus 7000 last week at the Data Center VT

This tactic address one key failing of ISCSI. Specifically an ISCSI interface running at 400 megabit will take 16% of a 3 ghz 64 bit cpu while using a software initiator or target. This can create some interesting issues where you don’t want to have them. By removing the layer3 information, and transposing fibre channel commands over the Ethernet transport a major cpu hit is avoided.

My gut feel is that this technology will follow a similar track as ISCSI did. ISCSI had a very low adoption in the first 3 years. Storage guys are naturally timid, as there are stiff consequences for failure, but eventually engineers warmed up to ISCSI and started deploying it where it was most appropriate, in lower bandwidth utilization hosts. FCOE will follow this same path, however this time we won’t have to fight the FUD of storage of a network transport. What is in FCOE’s favor is that as a protocol it is optimized for the general ISCSI setup of flat layer two transport between target and initiator.

That all being said, we are going to have to see some product releases from Cisco and Intel for FCOE to really take off. If we don’t, then FCOE will fall down the same hole that ATA over Ethernet disappeared down.

Colin McNamara
CCIE #18233 (Storage Networking)
2 Cups Solutions
“The difficult we do immediately, the impossible just takes a little longer”

References -

Intel Project Page

Colin McNamara
CCIE #18233

http://www.2cups.com

“The difficult we do immediately, the impossible just takes a little longer.”Similar Posts:

• Nexus 5020 – Consolidated 10 Gig Ethernet and 4 Gig Fibre Channel
• Cisco Nexus 4000 Blade Switch
• Link Round Up – L2TPv3 FCOE Trill Wounded Warriors
• My CCIE Storage Shopping List
• What being married to a geek who does search engine optimization gets you
• Will Cisco succede where Sun has failed?

--Colin McNamara

Fibre Channel over Ethernet is taking off

Partner List

Sort By Select Sort Type Company Name Country

Copyright ©2008 | Colin McNamara | CCIE 18233 | All Rights Reserved”

Colin McNamara
CCIE #18233

http://www.2cups.com

“The difficult we do immediately, the impossible just takes a little longer.”Similar Posts:

• Storage partners nationwide – 14
• How close did the fire come?
• Study on Perspective – Set 1
• Taking a closer look
• Why can’t I post from docs.google.com to blogger beta?
• Gall stones update – Kaiser sucks

--Colin McNamara

Cisco Storage Partners – California… a very small list