I am having a problem it's taken me a week to track down. I finally figured out how to use the top command to properly identify programs that are running when my server pins itself and becomes unresponsive.
I'm using the Turnkey LAMP 12 install, running on Amazon AWS EC2.
It appears that one of my Cron jobs is being executed off schedule, and multiple (10!) instances of this same PHP process were getting kicked off at the same time. What could cause this to happen? When this occurs, my server gets pinned with 100% usage until I restart the server, or today what I did was manually kill each runaway process... which instantly returned the server to normal levels.
The reason I believe that this is the cronjob getting run off-schedule is because the only other way this PHP script runs is when I manually kick it off. It's also being executed exactly as I have it spelled out in the cron job webmin page for this job.
The times that this stuff kicks off seems totally random. Earlier this week I had it happen at aproximately 10pm, and today it happened at 4pm. The job that it's calling multiple times is scheduled to run ONCE daily at 6pm.