GoDaddy, one of the nations largest registrars and hosting providers is distributing bloated and possibly insecure code that will cause the average user to more then double their hosting costs. In the pages below you will learn exactly what GoDaddy is doing to your server, how their support staff will try to upsell you, and the steps you need to take to ensure proper operation of your Virtual Dedicated Server.

Background

One of my new years resolutions this year was to consolidate hosting accounts into one virtual server (hosted). I had my domains, and my old hosting with Godaddy already so it was a no brainer to try out one of GoDaddys VDS (Virtual Dedicated Servers).

I went with their 29.99 a month package, with Centos5, unlimited domains, 10Gb disk, and 256 MB of memory. This should be perfectly fine for hosting a couple MySql driven sites, and a couple gallery instances. Let me emphasise this is only handling 4 active domains, two of which only have static HTML.

Provisioning was a breeze, from order to shell account only took 4 hours. I was provided with shell access, pre-configured yum repositories, and this web control panel – simple control panel, or TurboPanel (seems to have two different names). I was able to pop into GoDaddy’s control panel interface with a direct link from their server manager console, and was setting up domains in no time. (Let me throw this caveat out though, don’t buy this product for your mom’s hosting…. the documentation is horrible, and by horrible I mean non-existant).

So I get my server all set up a couple weeks ago, transfer all my files from 2 Cups Solutions and set up my new www.colinmcnamara.com site. Things go just fine, I changed over to wordpress as a CMS and am totally thrilled. My applications and email are working perfectly. Plus, I have a shell account at GoDaddy which is a very handy thing to have as a network engineer. Things are going so well, that I give my buddy Rick a Christmas present and get ricksdavis.com and el-cinco.net for him, and host it on my GoDaddy VDS.I go ahead and purchase the domain through godaddy’s domain manager. This is obviously GoDaddy’s core competency, and goes flawlessly as usual. Next step, I go into the Turbo Panel web interface. Let me give you a little background on TurboPanel. This is the “free” equivalent “to plesk or cpanel. It is actually ok for automating your domain provisioning, though if you are a normal skill level user, I would recommend paying the extra 9.99 a month for Cpanel.

Server Error

I open up my TurboPanel interface and go to provision ricksdavis.com into the domain manager. It comes back with the least descriptive error I have ever received.

Server Error
We are sorry, the system has encountered an error while processing your request.

Home

If you continue to receive this error, please contact your system administrator.

Your URL: /domain/edit.do

Error details:

CommandFailedException: Unable to get min/max uids
at c.g.t.f.systems.user.LinuxUserSubsystem.loadUids:825
at c.g.t.f.systems.user.LinuxUserSubsystem.getMinUid:780
at c.g.t.f.systems.user.LinuxUserSubsystem.loadUserInfo:670
at c.g.t.f.systems.user.LinuxUserSubsystem.getUserInfo:646
at c.g.t.w.actions.domain.ActionDomainEdit.process:84
at c.g.t.w.actions.AbstractSpringAction.execute:118
…
at c.g.t.w.filters.AuthorizedResourceFilter.doFilter:38
…
at c.g.t.w.filters.RequestPopulationFilter.doFilter:117
…

This is the most descriptive error ever right? it tells you what is wrong, has a link to the support system, and gives you actionable information…. I would say a resounding NO. This is a classic example of why friends don’t let friends program in Java. When I got this error last night, I was scratching my head. As an engineer, the first thing I will look at is the last change to the system. Coincidentally I had installed awstats two nights before, and looking at my change logs, I saw that I had upgraded my perl version. So, with no fast response to the support email from GoDaddy I chose to put in a server re provision request (fully automated) and restore from my backups. That process took about an hour, but afterwards I was back online with no errors. Eureka! I found it (I thought). I provisioned Rick’s domain, wordpress, gallery2 etc and then went to bed.

GoDaddy Support Response

Fast forward to this morning, and I finally recieve an email reply from godaddy support. The email is pasted below-

(I have replaced the agents name with John Doe. Tech support is a hard job and I see no reason to highlight him specifically)

Before re-provisioning the server, I went ahead and tried the old three finger salute (reboot) the error still existed. So even if this email would have came to me on time, it would not have helped.

But that is besides the point. lets dig into the solution

The agent suggested the following fix -

Restart tomcat – which I was NOT using for any of my web applications (not an ejb guy)

/etc/init.d/tomcat55 restart

Restart TurboPanel (or simple control panel now – they haven’t updated their init scripts)

/etc/init.d/turbopanel restart

GoDaddy tried to upsell me, instead of fixing their code

He gives the standard, run less stuff on your server speech (remember, I only have 4 domains on this server)

Now here is the kicker – To remedy this issue long-term, you could either setup a server with 512mb RAM, or upgrade to a Dedicated server.

What the heck is with that? I should not need an upgrade with only 4 domains on a server. Especially when in the setup the default Cpanel implies support of 30 domains. Is this Tech Support or a Sales Call?

But, since my server was working fine I don’t pay much attention to the email and move on with my life.

Fast forward an hour, and I am show Rick how to access all the features of his new site, and I figure that I need to change an email account on his domain to forward to his old account. Fine, this should take two minutes. I log onto the TurboPanel interface to put the email forward in.. and there it, a big useless error screen. What the heck is with that?

So this time I actually read the email, and try the fix. Results = nothing. GoDaddy offers shell access so I log in, run top and filter for %memory used. Low and behold, there is only 8142 bytes of memory free, and a Java process owned by root is using 300Mb of virtual memory, and 132Mb of real memory, and Tomcat is using 115Mb of virtual memory and 86Mb of real memory.

Let me translate this into English – Godaddy’s control panel application was using 218 Megabytes of the 256 Megabytes of memory I had purchased. That left me with 34 Megabytes of memory . Let me clarify this, I had paid for a virtual server with 256Mb of ram, up to 1000Mb bursted (which I think is their code for swap). I am only running 4 domains on this server, and two pop3 email servers. This should not be a problem.So what is the cause of the problem? I can sum it up, crappy Java programming. Someone decided to write this program in Java (probably easier to outsource) instead of optimising it to run on lean systems. Their code effectively takes up all the available memory. And on top of that, they are are running a webserver process as root… yes as root. It is like asking for your server to get hacked.

Now, that I am done ranting, let me highlight how to fix this problem.

If you are on Windows use the following procedure to get shell access to your GoDaddy VDS -

• you will want to download a ssh client called putty – Download Here
• Copy this file to your desktop, double click putty.exe , and you should see something like this -

• In the host name field I have www.yourdomain.com . replace yourdomain with your domain name.
• Click on the open button on the bottom right, and a shell should pop up, along with a warning that looks like this (you can click OK to the warning)

• Now skip past the linux section

If you are running Linux or Unix start here -

• Open up a command line terminal
• ssh using your godaddy simple control panel username example -

ssh your-godaddy-control-panel-username@www.yourdomain.com

• If this is your first time connecting to this server, you will be prompted to accept an unkown ssh key into known_hosts, choose yes to accept

Both Windows, Linux and Unix Continue Here –

• Enter your the password you use to access your GoDaddy Simple Control Panel Interface
• You will be presented with what looks like a DOS window, this is called a secure shell terminal. Type in the username you use to access your Godaddy Simple Control Panel and then hit enter

• Now type in the password that you use to access your Simple Control Panel Interface and hit enter

• Congratulations, if you see the window below you are now shelled into your virtual dedicated server.

Change to the Root user

• Now that you are shelled into a Linux device you need to escalate your privileges to get administrator level access. In the Unix world this user is known as Root. You can change to this user, and get full system privileges by using the following command.

su – root

• You will be presented with a password prompt, enter in the same password that you have used to log into your Simple Control Panel Interface
• You are now root, be careful with what command you enter under this user, as you can do some damage if you are careless

Clean out your servers memory

• Most recent Redhat direvatives (including Centos) utilize a tool called yum to add and remove packages. This is also true with your linux servers at GoDaddy.
• you need to install a tool called memhog, it is part of a package called numactl that is normally used to assign specific process’s to specific cpu’s in a multi-core system. We will be using it today to fix GoDaddy’s memory hogging application

yum install numactl

• choose yes to all the prompts, and numactl will automatically be downloaded and installed on your server

Stop GoDaddy Simple Control Panel, and Tomcat, and tell them not to start automatically when your server restarts.

• Tomcat is a special type of webserver for Java based applications. Godaddy uses it to run their control panel interface. 99.99999 percent of users will not need to use Tomcat. If you do need to use Tomcat then you are a technical user and will know what to do.
• In Linux, server applications are called daemons. The are executed by init scripts. We will use these scripts to turn off these server applications
• Turn off the TurboPanel daemon ( this is the process that runs your simple control panel web interface

/etc/init.d/turbopanel stop

• Next we need to turn off Tomcat

/etc/init.d/tomcat55 stop

• Now that we have these services turned off, we need to make sure that they don’t come back when we reboot the server. We can do this by using the chkconfig command.
• Stop the Simple Control Panel Interface from starting automatically by executing the following command

chkconfig turbopanel off

• Stop the Tomcat server from starting automatically by executing the following command

chkconfig tomcat55 off

Clean the mess GoDaddy made of your servers memory

• A couple steps back we installed numactl. The executable we wanted out of this package is memhog. Issue the following command to take your memory back. This command will overwrite 200 megabytes of your memory, allowing the rest of your applications to get access to that memory.

memhog 200m

• This will Clear out the memory that GoDaddy’s application took over, and allow the rest of your daemons to run fine.

Great, my server is running better now. But I want to use my Simple Control Panel Interface. How do I do that?

• Easy, all you need to do is temporarily start the turbopanel daemon. When you are done making changes, you can turn it off again

/etc/init.d/tomcat55 start

/etc/init.d/turbopanel start

• When you are done, don’t forget to turn it off

/etc/init.d/turbopanel stop

/etc/init.d/tomcat55 stop

*** UPDATE *** *** UPDATE ***

I managed to stay with Godaddy and use their Virtual Dedicated Servers for 12 months. However I have moved to a new hosting provider (rimuhosting) in the spring of 2009. My last straw with GoDaddy occurred when I noticed that even with my memory utilization under control my applications were performing poorly. I shelled into verify and found that the virtual disk I/O was severly limited.

Performance issues –

Simple command to determine directory size on GoDaddy VPS

[root@ip-72-167-15-128 home]# time du -hs
1.3G    .
real    0m49.013s
user    0m0.011s
sys    0m0.062s

Simple command to determine directory size on Rimuhosting VPS
[root@colinmcnamara home]# time du -hs
1.3G    .
real    0m0.343s
user    0m0.050s
sys    0m0.290s

Translated into english, it took 49 seconds to execute a simple command on my GoDaddy server, and it too 0.34 seconds (under 1 second) to execute the same command on my virtual server at Rimuhosting.

On top of that, I was paying $32 dollars a month to GoDaddy for a server with 256 megs of ram as well as a FTP backup account. When I created my short list of hosts, I noticed that I was not getting the best deal by staying with GoDaddy.

Pricing at Rimuhosting

• Virtual Dedicated Server 190 Megabytes RAM – $19.95
• Virtual Dedicated Server 400 Megabytes RAM – $29.95
• Virtual Dedicated Server 900 Megabytes RAM – $39.95
• Virtual Dedicated Server 1150+ Megabytes RAM – $49.95

One thing you may notice, is that for the same price as the base GoDaddy VDS ($29.95) you get 400 Megabytes of RAM instead of 256. (and they won’t put on a memory sucking turbopanel app) On top of that, things that I had to pay extra for at GoDaddy such as backup space, backup mail relays, and DNS service came bundled for free.

I made the choice to move to Rimuhosting, and the service has been superior to GoDaddy in every single way. If you get sick of GoDaddy like I did, you might want to check Rimuhosting out, I think you’ll like them.Similar Posts:

• What should I do this thanksgiving break?
• Zone based IOS firewalls
• Routers can email you when they go down
• New features in VMware 3.1
• Cisco NX-OS 4.0 | Next Generation Internet Operating System
• Identity aware networking using Cisco TrustSec

--Colin McNamara

Why GoDaddy Linux Virtual Dedicated Hosting Sucks & How to Fix It

My background is all over the place. It goes from running a small ISP when I was 18 to managing international CallManager clusters, to enterprise data center design and migrations. I swear that I have tech ADD. I see something new and I am like.. oooooh shiny, lets do a sniff and see how it works.

I actually attempted the Route Switch lab twice in 2002 (still had token ring and dlsw still) but got activated for the war before I could get my number. I can tell you.. that sucked horribly.

So, why did I choose to get my Storage CCIE first instead of finishing up my Route Switch first? Well, over the past couple years I have done a significant amount of IP storage (NFS, ISCSI, ATA over Ethernet, etc) including building my own IP storage heads based on linux. I had also done a little bit of fiber channel work. I felt that I had a significant advantage compared to most network engineers in the storage networking space. And, I think more importantly, the topics that I had to study were new and fresh. Whereas when I cracked open my Route Switch books.. I honestly wasn’t to excited about it at the time ( I think I was holding a grudge from my first attempts in 2002).

The partner e-learning central portion of Cisco has an Excellent lab access to labgear.net. It gives you 2 hour segments of time with 2 mds’s (both with IPS blades), 1 2 port jbod, and 2 2 port servers. They also have introductory tutorials for most major technology segments.

I did all of those, along with the every E-learning class that was on PEC. I think the major ones were the Design, Support, and CASSI classes.I took that, sat for the design and support specialist exams, and took my written last December.

After doing all that work, I was honestly 70% there. I took a little break for january and febuary (There was no open lab dates) and then started hitting the labs I think at the end of febuary.

For the majority of my practice I used the labs available through PEC. I also was able to weasel my way into 8 hour sessions every sunday from our channel SE. (I owe him plenty of drinks at networkers this year) For those 8 hour sessions I had labs created to summarize the major technology areas and to be as evil as possible to myself.

I had one attempt which didn’t work out as well as I would have liked. In RTP the lab starts at 7:15. This means if you are from the west coast like I am you will be getting up at 3:00 am in the morning for your lab. That royally kicked my but. I diverted from my attack plan and ended up
running out of time. It was a classic example of letting the lab run you, instead of you running the lab.

Luckily a data opened up just over a month later. I spent that time just working on my speed (speed is the secret sauce). I migrated to using Fabric Manager (gui interface) instead of command line. After 2 practice runs my time had dropped 25%. I also flew out to RTP 3 days before, and made sure to relax and get on east coast time.

For my final prep I got access to iementor’s lab. Their lab is excellent. I cannot reinforce enough how much it contributed to my confidence in that final week. Roman was really cool about working with an existing candidate to get me squeezed in. They are really cool guys, and they have the only workbook on the market right now.

So, I used their lab 2 days before mine, and then just chilled the day before. The day of the lab I had 70 points by lunch, and had completed configuration of the lab by 12:30. I spent the rest of the day reviewing my configurations (found 2 errors), fixing one bug, and generally harassing the proctor to make sure I didn’t misinterpret anything.

So.. if I had it all to do over again what would I do. I would still do the PEC stuff, and continue to primarily live in the Doc CD. Though I would probably pony up the cash and buy the iementor book and more lab time with them.

Copyright ©2008 | Colin McNamara | CCIE 18233 | All Rights Reserved”

Colin McNamara
CCIE #18233
“The difficult we do immediately, the impossible just takes a little longer.”Similar Posts:

• What does it take to pass the CCIE exam?
• Fibre Channel over Ethernet is taking off
• And it begins again – On the road to my CCIE in Storage
• Are you a kick ass engineer looking to grow?
• My CCIE Storage Shopping List
• Where was Colin in 2010?

--Colin McNamara

Why was Storage Networking my first CCIE? And What did I do to prepare?

For those who haven’t heard yet, I passed my CCIE Lab on June 14th. Now I can officially put CCIE #18233 after my name.

Anyone who has been down the same path understands how long of a road this is.
I started down this path in the spring of 1999 as the 432nd student ever in Cisco’s Networking Academy (which I promptly dropped out of to move to the bay area). I grew my networking skills all the way to passing my CCIE Written in 2001. I attempted my Route Switch lab in 2002, however I got called away to war for a year. Being away from it all for a year really cramped my style technically. In that time many of my certs expired, and I lost much of the momentum I had built of the past half decade.

Since I got back to the real world I had focused on work to the detriment of my certifications. I really did some great things, however I really neglected my resume.
In July of 2006 I had worked 2038 hours that year (for those not mathematically inclined, that is a full work year, in 1/2 a year) . At that point I decided it was time to stop neglecting my certifications. At that point I dedicated 8 hours each saturday, along with two nights a week to studies. By December of that year I had Certified on a few of the technologies that I had tons of experience in.

I managed to get my RHCE, Cisco Storage Support and Design Specialist, update my old IP Telephony specialist certifications to CCVP, get my Netapp Certified Administrator, and pass my CCIE Written for storage networking all by the end of the year. This spring I finished my Netapp Certified Expert and scheduled my Storage lab for early summer.

Anyone who knows me well knows how closely I track my time. That time tracking extends to my training. I tracked my training (reading, lab practice, testing, etc) just like any other part of my professional life. I spent around 150 hours studying for my Design, Support specialist certs, along with reading the recommended books of the CCIE reading list, and around 300 hours preparing for my lab exam. That is 300 hours configuring every possible combination and permutation of technology that could be setup, and then refining my speed in configuring those technologies until i got to the point where speed as well as brains would be an advantage in the lab.

So now that I have my CCIE, whats next? Well, oddly enough.. I am thinking of getting my second CCIE cert. In my office I will be the Jr guy by only having a single CCIE (on of our guys has all five). I also need to take my VMware certified professional cert, and probably get my HP Master Accredited Storage Engineer. I guess I am just a glutton for punishment.

Copyright ©2008 | Colin McNamara | CCIE 18233 | All Rights Reserved”Similar Posts:

• Where was Colin in 2010?
• About Colin McNamara
• Updated CCIE numbers
• Why was Storage Networking my first CCIE? And What did I do to prepare?
• And it begins again – On the road to my CCIE in Storage
• Update on the Cisco / Nuova connection

--Colin McNamara

What does it take to pass the CCIE exam?