Colin McNamara - CCIE 18233 , RHCE, GCIH, CCVP, GEEK

41 responses so far ↓

• 1 sketec // Feb 6, 2008 at 9:34 am

  Thanks for this. As my own blog gains popularity I might need virtual dedicated hosting soon.

• 2 colinmcnamara // Feb 6, 2008 at 4:12 pm

  Welcome dude, I am glad to be of help.

• 3 myfishorville // Feb 10, 2008 at 9:51 pm

  Thanks for the overview and solution. Ironically, I was dealing with this very issue tonight, wondering why GoDaddy support was telling me I was chewing through memory when I’m only hosting 4 domains, and really only 1 of them sees any traffic yet.

  I’m considering upgrading to cPanel, but I hate to put more money in their pockets.

• 4 colinmcnamara // Feb 11, 2008 at 7:21 am

  You are welcome. Cpanel is a very well developed product, and you wont be sad. You will effectively pay 120 a year for it ($9.99 a month).

• 5 Never Fear, Hosting Hero is Here! | Blogging Startup // Feb 13, 2008 at 8:25 pm

  [...] Colin McNamara - “Why GoDaddy Linux Virtual Dedicated Hosting Sucks & How to Fix It“ [...]

• 6 maxtoroq // Feb 16, 2008 at 7:18 pm

  Thank you so much for this post, I’ve been experiencing performance issues that lead me to order a second virtual server.

  I think you made a mistake at the begining of the post, the $29,99 month package comes with 256 MB of memory instead of 512.

  I would really apreaciate if you can explain me how to verify if the memhog command really worked, when I run the top command I don’t know how to read the stats at the because of the numbers, I don’t see 256 MB total, 180 MB used… I do know how to sort by %MEM, that gives me clear picture of what is using the most memory.

• 7 colinmcnamara // Feb 16, 2008 at 7:44 pm

  in the third row down in TOP there is a field called mem: with 4 values to the right. the third value is free.
  After running memhog you should see this number increase.

• 8 jstark // Feb 18, 2008 at 10:14 am

  I have had two godaddy vdeds for about 18 months. Overall I am happy with them, but I am very familiar with the terrible error messages provided by the SimpleControlPanel/TurboPanel. Normally, I just reboot the box and they clear.

  I am in the process of moving all the domains to one of the boxes so I can upgrade the OS on the older one and use it as a dev server. As I got more and more domains moved onto the newer box, I definitely noticed the page load times were getting slower. In fact, a lot slower.

  So, I ssh’d into the box, ran the top command, and made the exact same eye-popping discovery that you did - tomcat was eating all my memory! Unfortunately for me, I had no idea why or what to do about it.

  Thank you so much for this post. I followed your instructions to the letter and everything worked perfectly. My pages are now snappy as ever

  Best,
  j

• 9 colinmcnamara // Feb 18, 2008 at 10:57 am

  Jstark, you are very welcome.

• 10 JTBldrCO // Feb 18, 2008 at 5:33 pm

  Yes, great summary. Funny thing: I am a brand new GoDaddy VPS customer and I am in the 0.00001% that DO need Tomcat. I THINK I can reconfigure it to use less memory (launch-script java cmd line args for stack and/or heap, or, env var eg “export CATALINA_OPTS=”-Xms128m -Xmx128m”). But, I need a source for some help setting up my DNS. I have worked on the named.conf level years ago and don’t want to go back (tried to read OReilly BIND but muttled through at best). Any ideas to share for DNS HowTO ref. at the Simple Control Panel level?
  THANKS, JT

• 11 colinmcnamara // Feb 18, 2008 at 5:42 pm

  JTBldrCO - if you NEED tomcat, then I don’t see a problem in running it. ( I suggest turning it off because as you put it, 99.99999% of people will never use it. The root of the problem is a memory leak inside simple control panel, not tomcat.

• 12 jstark // Feb 19, 2008 at 12:35 am

  I just checked back in with my newly “memhogged” server and seems something is still awry.

  Earlier today when I followed your instructions, I saw my free memory skyrocket as expected — from about 50,000k to 200,000k.

  Now, it’s seems thing are back to bad. Here’s my Mem line from top command:

  Mem: 8145020k total, 8118540k used, 26480k free, 34056k buffers

  The weird thing is, tomcat is not running. In fact, when I sort the results by the %MEM column, only about 10 process are using any memory and they are all 0.3 or lower.

  I rebooted, but it didn’t change the situation. Any ideas?

  TIA,
  j

• 13 colinmcnamara // Feb 19, 2008 at 7:58 am

  Since the 2.6 kernel linux will start utilizing unused memory for buffer caching. This is fine,if it will give it up when another app requests it. GoDaddy’s app would not release those memory pages. You have 24mb free, which is fine. I was seeing that I only had 212 kb free (kilo vs mega bytes).

• 14 jstark // Feb 19, 2008 at 8:51 am

  Cool beans. Good to know. Thanks much for the explanation.

• 15 JTBldrCO // Feb 19, 2008 at 9:19 am

  I have successfully run the Simple Control Panel (SCP) with a changed memory setting. It turns out that, presently at least, the SCP runs in ITS OWN copy of Tomcat. The administrator-accessible Tomcat is a separate instance of this code. It shows up in the Processes as two separate ‘java’ commands. If you go to /etc/init.d and look in (file) turbopanel you’ll find “JAVA_OPTS” - it’s there that you set your “-Xms64m -Xmx96m” value (more or else - these are the values that I used). When I did that (a) it worked, (b) I saw lower memory usage when I ran top (this is the heap space used by the JVM - actual mem usage is higher than the mx setting). I am going to experiment with 64m/64m, too. Thanks for everyone’s help!

• 16 webcurry // Feb 24, 2008 at 1:31 am

  Thanks for this post Colin, helped me in ways you never intended:

  Decided to buy a new godaddy vds rather than re-provision my old one as they tried to get me to do (turns out I was paying 34.99 instead of 29.99 for the privilege of being such an old, loyal customer) with their 4 bullet point documentation.

  Opted for Centos5 rather than FC7 — just ’cause you did- and YOU know what you’re doing. Did check to make sure Plesk 8 supported it first… I’m not that lame.

  Followed your tutorial to learn how to have shell access. Yummy.

  All in all, a jolly good google story to tell the grandkids someday.

• 17 maxtoroq // Feb 24, 2008 at 1:43 pm

  Sorry to bother you Colin, but could you please explain more about the memhog command? Why do you override 200m? Does it have to do with the amount of memory that turbopanel uses or the amount of RAM the system has?

  For everyone who’s read this post, I’ve found this wiki that we could use to share more experiences about managing godaddy’s VPS:
  http://godaddyvds.wikispaces.com/

• 18 JTBldrCO // Feb 24, 2008 at 3:10 pm

  With respect to the TWO Tomcat containers running on the GoDaddy Virtual Dedicated Servers (at least, mine as I received it) - there is the standalone - Simple Control Panel application (see /etc/init.d/turbopanel), and there is the Tomcat container for us (me). Why they did not make TurboPanel a web app within the Tomcat container - well, guess it was for stability. Anyway, to set these containers’ JVM memory usage, you must see the two configs in /etc/sysconfig. These config settings take precedence over the settings in the files within /etc/init.d. Also, your -Xmx64m setting (64, 128, 48, whatever) is ADDITIVE to the PermSize setting to get actual heap space allocated (according to what I read elsewhere). You can see the settings in effect by doing a ps -ef with your shell window wide enough to output about 150 characters.

  See http://www.unixville.com/~moazam/stories/2004/05/17/maxpermsizeAndHowItRelatesToTheOverallHeap.html

• 19 colinmcnamara // Feb 24, 2008 at 4:15 pm

  JTBldrCO, thank you for your input. Your findings and suggestions will really help people who need to run tomcat applications on their GoDaddy Virtual Dedicated Servers, but don’t want to upgrade their memory.

• 20 colinmcnamara // Feb 24, 2008 at 4:29 pm

  maxtoroq - The choice to write over 200 megs of memory with memhog is purely based on the amount of memory provisioned in the system.

  The reason for memhog, is that with the 2.6 Linux kernel a feature was introduced that allowed the kernel to utilize unused pages in main memory to cache IO requests to connected block devices.

  This creates a tagging of pages in memory, tags for memory that can be over written and tags for pages that cannot be overridden.

  I don’t really know why, but with the move from 2.4 to 2.6 I have seen memory issues move from leaks requiring a reboot to memory getting tagged but not released. I have found that using memhog to “force” pages to be released works well to address this issue. I admit it is a total hack, but it has proved useful in a pinch.

• 21 topsax // Mar 14, 2008 at 8:40 pm

  Colin.. thanks so much for this article. I had the same problems you did with GoDaddy’s Virtual Server. I have been using it for 2 years now.. just seems to be a big problem with the C-Panel/Tomcat combination lately. Java just seems very lame to me. Never have liked it.

  I used your remedy.. it worked fine.. Now my little email programs are working again. When people write me about saxophones I have a small auto-responder. Of course like you said C-Panel running in the background was HOGGING all the memory.

  Anyway, just a few questions if you don’t mind?

  After installing “yum install numactl” I can’t seem to figure out how to get it to display the memory information again. I can’t figure out the commands.

  Also, the “memhog 200m” command worked the very first time.. but now the BASH says there is no such command when I try and run it. I am stumped??

  I don’t want Tomcat or C-Panel to restart automatically. I want to leave it off.. I tried running the 2 commands:

  chkconfig turbopanel off (and)
  chkconfig tomcat55 off

  Bash says again there are no such commands..

  Sorry to be a pain. If you can help I would appreciate it. I would like to be able to use these commands from time to time.

  YOUR WEB SITE ROCKS!!

  Cheers,
  Bob

• 22 topsax // Mar 14, 2008 at 10:15 pm

  Colin.. this is a follow-up to the above post.. I figured out how to use the CHKCONFIG command to turn off Tomcat and C-Panel..

  and I have a little better understand of YUM..

  Thanks!

  the only thing I can not figure out is how to run the “memhog 200m” command again. Maybe I am not in the right directory to run the command?

  Cheers,
  Bob

• 23 colinmcnamara // Mar 15, 2008 at 9:03 am

  Topsax, I am glad to hear that things are better after two years of this issue. Lets see if I can help you find memhog.

  First things first, make sure you followed the step in the above article that says to install it by issuing -

  yum install numactl

  this installs the package that memhog is distributed in.

  Next, if you have numactl installed, then you might have a path issue. Try executing memhog by explicitly calling out the path -

  /usr/bin/memhog 200m

  This should get you going in the right direction.

  –Colin

• 24 pagalkutta // Mar 25, 2008 at 3:17 am

  Hi
  Nice to see the comments on godaddy’s VDH and the particularly the problems people face.
  It makes me happy because now I know that at least it works enough to give problems.
  I got a VDH on the 20th of March and they told me it will be done in 24+5hours.
  Its been 5 days(120hrs+) and I can’t login to simple control panel.
  They say they are working on it and everytime I submit a trouble ticket and call them, they say it takes 72 hrs.
  I just finished submitting my 15th ticket and gave them a call. They say its going to be 72 hrs because I just submitted a ticket. Im gonna have to remember not to do that anymore.
  But im still positive about this. 30bucks for a VDS doesnt sound bad and they are open almost 24 hours although most of the people answering your calls can hardy use MS Paint.
  Cheers
  Ash

• 25 topsax // Mar 25, 2008 at 4:11 am

  I hope I don’t burst anyone’s bubbles.. with the Godaddy Virtual Server I finally had to strip it completely down.. ie: get rid of Java, Tomcat, and the Control Panel and NOW the server runs like a dream .. no RAM problems.. but, I am having to do everything in PUTTY SSH to configure or change things.. the good part is I am really learning linux.. the bad part is that it is time consuming..

  GoDaddy is a great company and the Virtual Server is a good inexpensive service, but in all honesty, the overhead of these operating systems and all the applications is just too much for the amount of RAM memory allocated on their plans..

  Sidenote: I just got a LINODE Virtual Server.. great deal at just 19.95 a month and PLENTY of RAM.. the Server really ROCKS.. but again, this is definitely not for beginners.. you really have to know linux to fool around here.

  Cheers,
  Bob

• 26 easytouch // Apr 9, 2008 at 2:50 pm

  I just wanted to leave my respect for your writing skills.

  It is not easy to write technical articles to be easy to read for beginners and also interesting for technical users.

  I hope that I can learn from your blog how to write good user manuals for my user audience.

  Greetings from Vienna, Robert

• 27 andybra // Apr 19, 2008 at 5:41 pm

  Thanks Colin I went through your item and its excellent, but there is one more thing that some people might need to be aware.

  When you turn off tomcat you affect cron jobs because they rely on Java to run. So you might need to keep tomcat running so your cron jobs wont be affected.

• 28 colinmcnamara // Apr 19, 2008 at 8:28 pm

  andybra, can you please elaborate on what you found. I checked my crontabs for each user including root and no java processes were listed (in the default go daddy configuration).

  Are you saying that cron requires java to run?
  I might be wrong, but I am pretty sure that I have had cron jobs running in the past without having installed java on a server (logratate, etc).

• 29 andybra // Apr 20, 2008 at 12:17 am

  I did a few tests, after I turned off my tomcat my cron jobs where not running, I tried for at least an hour, I have at least 12 crons running every minute. After I decided to just turn on the tomcat, my crons where working. I think the Crontab Manager requires Java.

  Is this right or am I missing something. How much RAM does tomcat take up also?

  Generally speaking my server is performing far much better after turning off and disabling the Simple control panel.

• 30 colinmcnamara // Apr 20, 2008 at 6:26 am

  andybra, remember that cron is just a scheduler for commands. If the commands that it is invoking need the tomcat process, and the process is disabled, then the job will fail. The litmus test for this is whether you can enter the command listed in your crontab manually. If you can’t run it on the command line, you will not be able to run it in cron.

• 31 doomtints // May 27, 2008 at 6:41 am

  This is actually a common problem with Virtuozzo VPSs. I hate to tell you this, but what GoDaddy told you is probably correct. Virtuozzo limits the number of open files you can have, and if you reach this limit no new processes can start causing a whole slew of errors. This is the most common support issue among VPS providers.

  From my experience, it is email that causes this problem for most people. If you are using your VPS for email, don’t POP very often and don’t run a mailing list.

  You can look at /proc/user_beancounters to see how many files you have open and also to see how many other resources you are using. The next time you have this error, look at this file and I bet you will see that you are over the resource limit for something.

  You claim to be a computer geek but this seems to be new territory for you. I’m not a godaddy fan by any means. so I’m not defending them, but with all your certifications this post makes you look like a nOOb.

• 32 colinmcnamara // May 27, 2008 at 7:52 am

  Doomtints, Thank you for your input on open file limits. You are correct that having to many open process or files can have negative effects on system performance, and that it is generally limited by the sysadmin. Though through my troubleshooting I had ruled that out as a culprit. I do have to say that I think you are jumping to a conclusion, and frankly being a bit rude.

  My troubleshooting was pretty specific, and my solutions have helped people with similar problems on their servers. I think that passes any reasonable persons litmus test for a correct solution.

  I would hope that you would learn that putting down someone else technically does not elevate yourself, it actually lowers peoples opinions of you. Everyone has to learn this lesson early in their careers, and once they do, that humility is a firm foundation build on.

  Also, certifications don’t make the man. I know some brilliant engineers with no certifications to their names, as well as amazing engineers who have more certs then they know what to do with.

  In my case certifications are a necessary part of consulting. They are important in that they prove at a base level that you understand that vendors product, as well as providing a way to learn more about a product or technology (yes, I actually learn something new every time I have to cert or recert on a product). If you question my experience on a product or technology, please feel free to check out my resume. You will find that I am just passing 10 years of experience with Unix and Routing / Switching technologies. While that doesn’t mean I know it all ( I learn something new every day), it does provide me with a decent base of knowledge and experience that I apply every day to learning and understanding new technologies.

• 33 doomtints // May 27, 2008 at 1:47 pm

  I don’t think I made any assumptions.

  - You keep saying you have a VDS, but with the specs I can see that you clearly have a VPS.

  - You still didn’t understand what I stated. Virtuozzo puts hard limits on the number of open files that can be open on a VPS, after that it won’t allow new processes to start. This is different from any “Linux” limit which I don’t think exists, but may of course limit performance if too many files are open. You would probably agree that what you ran into with your error messages isn’t a performance issue. Your response to me makes me think that you still don’t understand what I told you.

  Look at /proc/user_beancounters on your VPS and you will see exactly what I’m talking about. Very likely the plan you have is the standard $30 plan at most providers, which means you have an open file limit of around 8196. If you have ever used the ‘lsof’ command in Linux, then you know that a standard Linux installation can easily have 2000-4000 files open even when apparently idle. Add in a mailing list, a lot of email traffic, or a lot of setup/maintenance activity and you can see that you’ll be hitting that limit fairly quickly.

  - Sure, you helped people, but you didn’t fix the real problem that you were having or that they were having, as you haven’t yet identified the problem. Godaddy’s advice to you, which you have attacked, was better advice. Why are you attacking Godaddy’s advice but then a little offended when someone attacks yours? If someone is hitting the resource limit with the VPS plan they bought, the best advice for a service provider to give is to upgrade as opposed to disabling services. Sure, experts can disable services that they don’t need, but even then they are not “fixing” the problem with Virtuozzo! The problem with a hard limit on open files could still recur!

  Play a little bit with Virtuozzo or, better yet, OpenVZ (the free version) and you will find that you were off on the wrong track here.

• 34 colinmcnamara // May 27, 2008 at 2:26 pm

  Doomtints - I guess we will have to agree to disagree here. Open file limits (either hard limits internally, or imposed by an external virtualizer) were one of the first things I did check… this isn’t my first rodeo, while my shell scripting is horrible, I am not so bad when it comes to Linux / Unix. I feel that your condescending tone is not appropriate, nor is it helpful.

  I am glad that you were able to contribute to the conversation here, but the issue you are describing is not and has not occurred on my server. I run 3 word press driven sites, and one instance of a mail server. This is well within the bounds of what a virtual server should be able to handle.

  I could post every detailed debug, and change this site from a blog that tries to explain complex technologies in a voice that most people can understand into a site thats only purpose is to show the world that I am smart and they aren’t. Frankly I chose not to. If you disagree with that choice, the internet is a big place, and you are free to write your own articles with your own opinions.

• 35 WebSmith // Jun 19, 2008 at 9:39 am

  Well i’m an unabashed Linux ignoramus, although it’s something i’m learning bit by bit. I wonder if some people make things complicated because, i mean, if you can explain something like this to me then you can explain it to just /anyone/ and then who needs self-important experts? Anyway i, for one, very much appreciate the plain-English instructions and the lack of patronizing tone. This is a difficult paradigm for novices because the most frequent advice we hear is “well if you don’t know /that/ then you shouldn’t be running your own server”.

  I’m just a small-time developer who likes to provide hosting for my clients. I bought a Godaddy VPS because i needed one more IP for a client’s SSL cert, and that would’ve made my shared reseller account cost as much as a GoDaddy VPS. My first impression is that it’s a dog right out of the box, which is why i Googled for articles such as this one. It takes almost two minutes to connect to the FTP server, and randomly anywhere from ten seconds to a half-minute to change directories. Same behavior is exhibited in SCP.

  Using one test page with four images i can see significant difference in speed between the GoDaddy server and my overloaded old shared hosting. I’m really wondering if i did the right thing, and i’ve got a very mportant account waiting while i’m wondering where to install his certificate. When you speak of your troubles with “only four domains” are you giving an extreme example as a point of reference? I mean, i’ve got at least fifteen well-trafficked sites and two busy php forums… the specs would indicate i should have about ten times the resources of my shared server, but your server is throwing errors with four static sites… um… sorry… this is a question and i’m not articulating it well because i’m getting flustered.

  Sorry for the long post. I’m going now to check out the advice most graciously given here, but meanwhile would you have any advice as to how one might transfer files to and from the server via SCP or SFTP when everything belongs to “root” but you have to log in with your username? File operations are really clumsy from a PuTTY terminal.

• 36 colinmcnamara // Jun 19, 2008 at 4:36 pm

  Websmith, I am glad that you found my blog helpful. I do try to avoid the patronizing uber geek tone, I don’t find that it is very helpful in the long run.

  My for domains have moderate traffic, with around 8,000 views a month right now. Its nothing to laugh at, but small beans in the grand scheme of things.

  One tool that I recommend for windows users SFTP’ing data is winscp. It is very similar to common FTP programs and should be pretty easy to use for you.

• 37 luisw19 // Jun 21, 2008 at 7:21 am

  Great article indeed! Many thanks

  I do not consider my self a Linux geek, but I have manage to maintain my own VPS for about 5 years. Since I have not updated my VPS since I started I am probably paying the most expensive and less competitive VPS package on the market (I started with what used to be Interlard and was bough by web.com sucks!). Saying this, I am now in the process of choosing the new VPS, reason why I ended up reading this outstanding blog.

  I was wondering if you may please provide me with any hint as to whether or not goodady should I go with goodady VDS (I can assume you don’t like it too much, but honestly, is among the cheapest ones in the market reason why it results very attractive to many, including me!).

  I own about 20 accounts (domains) and offer to each account most common services such as email (pop3, imap, smtp and webmail), ftp, mysql; my concern if whether or not godaddy VDS offers all of this services independently to each domain (specially the email services… in other words, does it includes a mail server such as qmail and sendmail??). Most of the accounts do not have a lot of traffic, but one specifically is very somewhat popular and receives about 2000 hits a day! (all my apps are LAMP based!)

  On the other hand, I am also attracted to other VPS plans such as the ones offered in 1and1.com and linode.com! any feedback on these plans??

  Any help will be most welcomed!

  Cheers,

  Luis

• 38 placibo.var // Jun 26, 2008 at 10:53 am

  colin, thx for the info. I just got the same exact VDS service day before, and was running into the same problem.

  Disabling the memory hog fixed the server slowdown but its still serving pages slowly. Looks like theres some amount of bandwidth or connection throttling somewhere, or it needs optimization.

  I haven’t tried to run the same app on my server server at home. It is much more powerful but,

  Also, all the resource use were WAY under the limit. doomtints, pls brush up on ur attitude, and stop throwing commands out of your linux for dummies book.

• 39 placibo.var // Jun 26, 2008 at 10:58 am

  people like doomtints make me sick, bloody noob, script kiddie.

• 40 deltajo // Jun 26, 2008 at 6:00 pm

  Colin what a great find this blog was. Thank you very much.

  I encountered the exact same memory problem that you described (and got the same form letter it sounds like) and am now setting up my “new” 512 MB RAM virtual dedicated server, having fallen for their sales/tech answer to my problem.

  I’ve implemented your solutions on my old server and the new one. No probs so far!

  But do I understand correctly that if I changed from Simple Control Panel to another one of the options they offer, the memory problem won’t persist, and I can avoid this hassle of having to stop/restart through SSH all the time?

  Also, re the argument from the pretty rude “doomtints” (#31 above): One of the sites I have on my server gets thousands of hits *per day* (it’s a blog using Expression Engine), and I have a massive incoming spam email problem, in addition to lots of legitimate emails which I totally rely on and have to check constantly through MS Outlook Express. Could that guy’s remarks about email argue for a separate server for email?

  Thanks again Colin. Great work!

• 41 robert // Jul 8, 2008 at 8:30 pm

  Hello Colin

  I have started a dedicated server package there with GOdaddy….
  I cant find the tomcat root there ….can u suggest me where its present….I have logged in as root to the server with ssh.Yet could not find the root.

  tomcat is in /usr/java/tomcat

  where is the web-apps specifically?

  Would u suggest?

  Robert

You must log in to post a comment.