ccie, vcp ccvp, rhce, giac, gcih, cisco, netscreen, netscaler, juniper, f5, security, virtualization, vmware

OpenStack Nerd, CCIE, DevOps Junkie

Changing the world, one person at a time

OpenStack Nerd, CCIE, DevOps Junkie header image 2

Simplifying scale out DataCenter design with UCS Manager 2.1

November 1st, 2012 · 36 Comments · CISCO, Cisco UCS, Cisco Unified Computing System, Cisco Unified Computing System Managerm UCSM

I’ve been designing and deploying UCS since the product was released a couple years ago (technically I was involved in the pre-release so we will say since UCSM v 0.8). From the start I was constantly pushing up against scalability and design constraints of UCS. The benefits of the system outweighed the challenge, but these design constraints created some challenges in creating external systems to meet the needs of large UCS customers.

Don’t get me wrong, out of any server platform I prefer UCS. That being said there are a few area’s that have really caused headaches for me over the years.

Headaches solved with the release of UCSM 2.1


Headache #1 – Once I scale past a certain number of servers, I have to establish a new UCS domain

This has been a huge challenge for both large single data center instances, as well as multi data center instances (such as DR). In both these cases I would have to utilize tricks like placing mac address, wwn pools and other “unique” identifiers into a CMDB (Configuration Management Database) outside of UCS. And even when utilizing external CMDBs, there was a still a bit of design necessary to lay out UCS domains in a fashion that would support eventual integration in the future without overlapping configuration elements.

All of this work was done to ensure that if two servers were instantiated in two different UCS domains that they wouldn’t have conflicts if they wound up on the same segment. Handling this logically by bit swapping the UCS domain ID in certain resource pools wasn’t terribly complicated, but in my opinion unnecessary (though integration with CMDB’s can be very complicated).

This got even more complicated If you wanted to have a DR site. Making something simple happen like having a server that boots from SAN boot of the DR site SAN in an outage involved using external tools or scripts. In my opinion this is something that should be handled by UCSM or a manager of UCSM.

Headache solved – UCS Central Manager of Managers


For those in the now, this has been in the works for a VERY long time. In fact the early install (1000+ servers) that I mentioned above where we had to use external CMDB’s to glue UCS domains together in the first year of UCS generated this feature request. 

UCS Central is in a sense a manager of managers. This allows you to aggregate pools and policies of multiple independent UCS domains into one central management platform.

It solves the problems –

  • resource conflicts across pools,
  • mobility of service profiles between UCS domains as well as
  • centralizing access logs
  • centralizing access to console servers

Headache #2 – Even when Cisco released code to manage c-series 19″ rack mounts under UCSM it still required a bunch of extra cables and equipment to make it work.

70% of the worlds x86 servers are in a 19″ rack mount form factor. Recently Cisco enabled them to be managed under UCSM and to have a data path that exits through the fabric interconnects. This allowed a couple key things to happen. First, it allowed a unified view of systems for an administrative staff for a data center. Second, it allowed a clean data path from, say a storage caching engine run on a a b-250 blade, to a compute node housed on a c240 rack mount. All of this communication would be contained within the fabric interconnects, and not have to exit northbound as it had in the past.

I was happy with this release. It allowed the c-series servers to be managed under UCSM with the same tools techniques and API’s that we manage the blades with. However the code was not updated to allow all that magic to happen over a single wire.

You would end up with beautiful cabling on the backs of your blade centers, and a giant mess of cables coming out of your rack mounts since you needed separate cables to support data path vs management plane. Call me a neatnick, but I like my racks to be pretty and clean (and not have to buy extra switches, cables and adapters).

Headache solved – Single wire management for ALL UCS servers


With the 2.1 release now all you need is a single Cisco Virtual Interface Card in your UCS 19″ rack mount (two if you want redundancy) to allow the full feature set that you have available on a UCS blade. For me this not only simplify’s my designs, but also allows flexibility in things like designing Hadoop and OpenStack Swift Object Storage clusters where redundancy is done at the application level and dual 10 gig interfaces are not needed.

Headache #3 – For certain topologies I want storage locally attached to my pods using whatever protocol I want

Here is a dirty little secret. Even though you can abstract a bunch of storage functions into UCS, most server guys are still a bit impatient with their peers on the storage teams. There are many times when the server guys want to consolidate a bunch of boot disks into an array and connect them directly to the fabric interconnects.

Over time Cisco has been releasing support for additional protocols connected in this way, however it was not ubiquitous. This created problems because you could not create a standard topology that supported flexible protocol consumption in your network. You would end up with two to three variants of supported topologies. In my opinion this creates issues with operational procedures and leads to extension of outages and generally inefficient designs.

Headache solved – Flexible and consistent storage topology options no matter what protocol is being used.

ucs fcoe multi-hop cisco

With UCS 2.1 now, no matter what protocol floats your boat you can implement them in a consistent manner. This may include directly connecting Fibre Channel storage to your fabric interconnects and zoning them. Or it may include utilizing multi-hop FCoE (I’ll leave the argument to whether you SHOULD use this till later).

Either way, the most important thing to me is that no matter what the design requirements are. Now you have the tools available to meet them in a consistent fashion without changing your entire network and systems topology.

Colin’s Thoughts

Quite often there is lots of glitz and glamor when a new product is released. Press conferences are held where everybody looks at the shiny blinky things and oohs and aweess. However when new software makes things you already use every day work better, or allows them to do new things comes out nobody notices.

In this case the 2.1 release of UCSM takes a product that many people already have (Unified Computing System) and makes it do more. There aren’t going to be press conferences about this, but it is worth taking a closer look at. It will make my life easier, and I hope it does the same for you.




Tags: ···············

36 responses so far ↓

  • 1 Cisco Announces Updates to UCS and Nexus Lines // Nov 1, 2012 at 6:02 pm

    [...] us to do Fiber Channel zoning directly on the Fabric Interconnects as well as multi-hop FCoE.  Here’s a post by Colin McNamara that talks about both UCS Central and UCSM [...]

  • 2 Will Hgan // Nov 2, 2012 at 5:03 am

    Great post on UCS.

    Somehow this reminds me of when Cisco came out with that Call Manager manager of managers (I cant remember the product name).

    Really hope you follow up on FCOE comment:
    “I’ll leave the argument to whether you SHOULD use this till later”

  • 3 Cisco UCS Central :Scaling Is a Necessity. Adding Cost and Complexity Is Not. - WKSB Solutions // Nov 6, 2012 at 8:37 am

    [...] McNamara about the 3 major “headaches” solved by the introduction of UCS Central and UCSM 2.1 [Simplifying Scale Out DataCenter Design with UCSM 2.1]“The 2.1 release of UCSM takes a product that many people already have (Unified Computing System) [...]

  • 4 party planning ideas // Jun 16, 2013 at 9:38 am

    Nice post. I was checking constantly this blog and I am impressed!
    Very useful info particularly the ultimate section
    :) I maintain such info a lot. I used to be looking for this certain
    info for a very long time. Thank you and good luck.

    Look at my web page; party planning ideas

  • 5 Lindsay // Jun 17, 2013 at 7:25 am

    Thought you might want to learn about our new single fiber X2 transceiver. Might be of interest to your readers.
    Let me know what you think:

  • 6 elephant birthday party Supplies // Jun 19, 2013 at 2:37 pm

    I have been browsing on-line greater than 3 hours nowadays,
    but I never found any attention-grabbing article like yours.
    It is beautiful value enough for me. In my opinion,
    if all site owners and bloggers made just right content material as you did, the
    web will likely be a lot more useful than ever before.

    My blog post: elephant birthday party Supplies

  • 7 land financing // Jun 23, 2013 at 1:31 pm

    I have been browsing online more than 4 hours today, yet I never found any interesting article like yours.
    It’s pretty worth enough for me. In my opinion, if all site owners and bloggers made good content as you did, the internet will be much more useful than ever before.

    Here is my blog – land financing

  • 8 personalized bridal shower // Jun 26, 2013 at 2:53 pm

    I think the admin of this website is really working hard for his site, since
    here every information is quality based information.

  • 9 // Aug 4, 2013 at 3:33 am

    It’s awesome to pay a visit this web page and reading the views of all mates about this piece of writing, while I am also keen of getting experience.

  • 10 commander viagra acheter // Mar 28, 2014 at 5:07 pm

    I am sure this article has touched all the internet visitors, its really really nice paragraph on building up new web

  • 11 Learn Additional // Apr 2, 2014 at 8:04 pm

    Sounds simple but we aren’t generally doing it in the United States.
    Me: How do you feel about petfinder like a service to help dogs find forever homes.

    Page numbers ought to get incorporated inside upper left half of the header.
    , it could possibly confuse an individual, once you actually mean a similar
    link Learn Additional You will likely be notified as you might be typing the ‘allow links’ in your article.

    It can be less stressful to write by hand in case you have poor typing skills or you never actually enjoy using a

  • 12 // Apr 10, 2014 at 11:05 am

    It is really a great approach to get lots of exposure and post ads several times in various ways.
    We feature latest and detailed info on Barrington homes for sale that can help you to make your decision to get a home easier.

  • 13 games in mobile // Apr 14, 2014 at 3:37 am

    Hey There. I found your blog using msn. That is a very neatly written article.
    I will be sure to bookmark it and come back to read more of your
    helpful information. Thank you for the post.
    I’ll certainly comeback.

  • 14 Mary W. Herrington // Apr 29, 2014 at 5:44 am

    Genuinely motivating points you’ll have mentioned, thank you so much for writing.

  • 15 Michael K. Erdman // Apr 29, 2014 at 5:44 am

    I just need to reveal to you in which I am certain inexperienced to having a blog and utterly admired your work. Very possible I am going to bookmark your article post . You really have wonderful article text. Like it for giving out with us your web post.

  • 16 Margaret C. Crowe // Apr 29, 2014 at 5:44 am

    Very beneficial points that you have remarked, thank you for putting up.

  • 17 Johnd505 // Apr 30, 2014 at 8:03 pm

    The data are part of an autumn of just about 16 targets in Lib Dem provide because of the 2010 commander selection, before Huhne scooped 46.5 with all the electionHermes kelly felix bags didn can far in order to move benefit ddfaedfkaged

  • 18 Timothy S. Lester // May 8, 2014 at 5:41 pm

    Was looking throughout . I appreciate your concepts. Do write up much more of this theme ok. Thank you

  • 19 Michael T. Anderson // May 8, 2014 at 10:26 pm

    I observe there are several junk mail on this blog site. Are you thinking of intending to clean it up. Perhaps I can assist to complete?

  • 20 Kevin R. Harvey // May 8, 2014 at 10:44 pm

    We merely have to tell you that I am certain beginner to putting up a blog and really valued your webpage. Likely I am most likely to store your blog post . You literally have fabulous article content. Acknowledge it for share-out with us your internet site write-up.

  • 21 clumsy ninja hack.rar // May 26, 2014 at 6:51 am

    Aw, this was an exceptionally good post. Finding the time and actual
    effort to generate a good article… but what
    can I say… I hesitate a lot and don’t manage to get anything done.

    My blog :: clumsy ninja hack.rar

  • 22 7 zip free download // Jul 14, 2014 at 5:50 pm

    Incredible points. Outstanding arguments. Keep up the amazing spirit.

    Feel free to surf to my web-site – 7 zip free download

  • 23 mpomponieres vaptisis // Jul 20, 2014 at 4:21 am

    mpomponieres vaptisis I’ve very little expertise in computer programming however I had been hoping to start my own blog in the near future. Anyway, if you have any ideas or techniques for new blog owners please share. I know this is off subject but…

    I’ve very little expertise in computer programming however I had been hoping to start my own blog in the near future. Anyway, if you have any ideas or techniques for new blog owners please share. I know this is off subject but I just had to ask. Thank…

  • 24 norton internet security 2014 serial key // Aug 2, 2014 at 8:06 am

    Spot on with this write-up, I absolutely feel this website needs far more attention. I’ll probably be returning to
    read more, thanks for the info!

    Look into my webpage; norton internet security 2014 serial key

  • 25 Jc // Aug 3, 2014 at 11:32 am

    It is a journey that must be made for personal happiness but I have
    never met a transgender who hadn’t suffered greatly from the journey.

    The individual can first of all think about shaving.
    In 2010, Abi had her FFS in Marbella, Spain, which has
    significantly improved her ability to pass in public and therefore her self-esteem.

  • 26 leadpages system Review // Aug 14, 2014 at 6:11 pm

    My spouse and I absolutely love your blog and find a lot of your post’s to be
    exactly what I’m looking for. Does one offer guest writers to
    write content for yourself? I wouldn’t mind producing a post
    or elaborating on most of the subjects you write about here.
    Again, awesome site!

  • 27 medzilla Job Search // Aug 14, 2014 at 8:48 pm

    I know this if off topic but I’m looking into starting
    my own blog and was wondering what all is needed to get set up?
    I’m assuming having a blog like yours would cost a pretty penny?

    I’m not very internet savvy so I’m not 100%
    sure. Any tips or advice would be greatly appreciated.

    Appreciate it

    my weblog: medzilla Job Search

  • 28 Russell // Aug 21, 2014 at 7:57 am

    Ideal Search and rescue dog is really a instruction collar which has affordable rates.
    You do not understand the value of hiring check out
    here this professional in explaining the rewards
    of training in 2012 can take a seat. First of all dog trainers and training,
    you could check with dog.

    Here is my blog; (Russell)

  • 29 Barbara // Aug 21, 2014 at 7:58 am

    Dogs use” here’ puppy training when your spouse uses” come” and”
    here” or” LEAVE IT” STOP! Then it can be changed by the dog tells your dog the kind of training there. Some people feel that punishment given is to jump up on somebody, and do whatever they want to train their dogs.

    my web page :: (Barbara)

  • 30 seo // Aug 23, 2014 at 9:53 am

    Never sound demanding or brag about how great your website, products
    or services are and how the person you are asking would
    benefit from granting you this favor. Submitting
    articles is advantages are numerous, you get credibility, fast visibility, traffic and one way links back to your website.
    More and more reciprocal links can be made from domain name
    for site.

  • 31 domain // Sep 2, 2014 at 8:22 am

    I rarely drop remarks, but after reading through a few of the remarks
    here Simplifying scale out DataCenter design with UCS
    Manager 2.1 | OpenStack Nerd, CCIE, DevOps Junkie.
    I actually do have 2 questions for you if you do not mind.

    Is it only me or do some of the comments look as if they are coming from brain dead visitors?
    :-P And, if you are writing at additional sites, I’d like to keep up with anything fresh you have to post.

    Could you list of every one of your social community sites like
    your twitter feed, Facebook page or linkedin profile?

  • 32 The Latest Insights Into No-Nonsense Family Guy The Quest For Stuff Hack Strategies // Sep 5, 2014 at 10:05 am

    If you family guy the quest for stuff hack search for this.

    Large space needed family guy the quest for stuff hack for safe driving.

    An online Entertainment portal offering complete mobile applicationslike
    free downloadable mobile software, Mobile ringtones, mobile games one can be one significant part found in this field with several amazing offers.
    The mobile gaming industry do more than 400
    games download mobile games creates curiosity in users mind and tired body.
    Let s enjoy your time by browsing through our own tests and ratings.

    Feel free to visit my site The Latest Insights Into No-Nonsense Family Guy The Quest For Stuff Hack Strategies

  • 33 racing rivals cheats // Sep 13, 2014 at 3:56 am

    95Multi-player, multi-table racing rivals cheats hold’em.
    These are nothing short of the sites.

  • 34 samurai siege hack // Sep 17, 2014 at 6:48 am

    She’s been operating on her web samurai siege cheats site.
    If you look at next analysis of NFL Pro 2012 for your iPod, iPhone, iPad,
    iPod Touch, you’ll be getting a complete debut of Superstars fighting styles and technique.
    It is not limited and are beginning to check out the databases for your phone,
    smartphone, PDA or hand held mobile phone gaming. What is the fascination with mobile phone is
    the first to make some money.

    Feel free to visit my web page; samurai siege hack

  • 35 bike race hack apk // Sep 23, 2014 at 3:17 pm

    This is because of obsessively playing video games on your
    phone with you, but from different online websites and find your lost phone on standby to place ads in context with the wonderful monster designs.
    Using the Animus 2. Mobiles will be suitable for a while and can be easily jailbroken allowing the provision of various
    bike race hack units indirectly. 1 or above and giant spiders on the market.
    The next edition bike race hack of the best apps with minimal advertising.

    Also visit my site bike race hack apk

  • 36 search engine // Mar 21, 2015 at 12:48 am

    I enjoy, result in I found just what I was looking for.
    You’ve ended my four day long hunt! God Bless you man. Have a great day.

Leave a Comment