Front page
Silflay Hraka?

Bigwig is a systems administrator at a public university
Hrairoo is the proprietor of a quality used bookstore
Kehaar is.
Woundwort is a professor of counseling at a private university

The Hraka RSS feed

bigwig AT

Friends of Hraka
Daily Pundit
cut on the bias
Meryl Yourish
This Blog Is Full Of Crap
Winds of Change
A Small Victory
Silent Running
Dr. Weevil
Little Green Footballs
Fragments from Floyd
The Feces Flinging Monkey
the skwib
Dean's World
Little Tiny Lies
The Redsugar Muse
Natalie Solent
From the Mrs.
The Anti-Idiotarian Rottweiler
On the Third Hand
Public Nuisance
Not a Fish
Electric Venom
Skippy, The Bush Kangaroo
Common Sense and Wonder
Neither Here Nor There
The Greatest Jeneration
Ipse Dixit
Blog On the Run
Redwood Dragon
Greeblie Blog
Have A Cuppa Tea
A Dog's Life
Iberian Notes
Midwest Conservative Journal
A Voyage to Arcturus
Trojan Horseshoes
In Context
The People's Republic of Seabrook
Country Store
Blog Critics
Chicago Boyz
Hippy Hill News
Kyle Still Free Press
The Devil's Excrement
The Fat Guy
War Liberal
Assume the Position
Balloon Juice
Iron Pen In A Velvet Glove
Freedom Lives
Where Worlds Collide
Knot by Numbers
How Appealing
South Knox Bubba
Heretical Ideas
The Kitchen Cabinet
Bo Cowgill
Raving Atheist
The Short Strange Trip
Shark Blog
Ron Bailey's Weblog
Cornfield Commentary
Northwest Notes
The Blog from the Core
The Talking Dog
WTF Is It Now??
Blue Streak
Smarter Harper's Index
nikita demosthenes
Bloviating Inanities
Sneakeasy's Joint
Ravenwood's Universe
The Eleven Day Empire
World Wide Rant
All American
The Rant
The Johnny Bacardi Show
The Head Heeb
Viking Pundit
Oscar Jr. Was Here
Just Some Poor Schmuck
Katy & Bruce Loebrich
But How's The Coffee?
Roscoe Ellis
Sasha Castel
Susskins Central Dispatch
Josh Heit
Aaron's Rantblog
As I was saying...
Blog O' Dob
Dr. Frank's Blogs Of War
Betsy's Page
A Knob for Brightness
Fresh Bilge
The Politburo Diktat
Drumwaster's rants
Curt's Page
The Razor
An Unsealed Room
The Legal Bean
Helloooo chapter two!
As I Was Saying...
SkeptiLog AGOG!
Tong family blog
Vox Beth
I was thinking
Judicious Asininity
This Woman's Work
Fragrant Lotus
Single Southern Guy
Jay Solo's Verbosity
Snooze Button Dreams
You Big Mouth, You!
From the Inside looking Out
Night of the Lepus
No Watermelons Allowed
From The Inside Looking Out
Lies, Damn Lies, and Statistics
Suburban Blight
The SmarterCop
Dog of Flanders
From Behind the Wall of Sleep
Beaker's Corner
Bad State of Gruntledness
Who Tends The Fires
Granny Rant
Elegance Against Ignorance
Say What?
Blown Fuse
Wait 'til Next Year
The Pryhills
The Whomping Willow
The National Debate
The Skeptician
Zach Everson
Geekward Ho
Life in New Orleans
Rotten Miracles
The Biomes Blog
See What You Share
Blog d’Elisson
Your Philosophy Sucks
Watauga Rambler
Socialized Medicine
Verging on Pertinence
Read My Lips
The Flannel Avenger
Butch Howard's WebLog
Castle Argghhh!
Andrew Hofer
Moron Abroad
White Pebble
Darn Floor
Pajama Pundits
Goddess Training 101
A & W
Medical Madhouse
Slowly Going Sane
The Oubliette
American Future
Right Side Redux
See The Donkey
Newbie Trucker
The Right Scale
Running Scared
Ramblings Journal
Focus On Reality
Wyatt's Torch

April 30, 2003

Tricks of the Trade

Just had to kill multiple Java threads spawned by a hanging database query, or something so similar to a hanging db query that it makes no difference. On Solaris, this would be comparatively simple, as all the threads would have been represented by a single process id. One kill -9 < pid > would have done the trick. On Linux, or at least the 7.2 Redhat version we run, each thread has it's own pid, so before I could restart the Tomcat servlet engine I would have had to kill each one individually. It's not difficult, typing "kill -9 < pid > " over and over again, but it is onerous, or at least as onerous as my job gets.

Here's where I tell you a secret. The best sysadmins are lazy bastards, so lazy that typing in even just 10 "kill -9 < pid > " commands seems an overwhelming task. I am not a great sysadmin; I'm not quite lazy enough*. Most of the time I'll just type in the multiple commands. Today I decided not to.

Here's what the processes that needed killing looked like; (I've broken them up so that they won't screw up viewers with small monitors. Normally everything in the 2 entries below would appear on one line)

root 18935 0.0 6.7 236880 86704 ? S Apr28 0:00
/usr/Java/jdk1.3.1_07/bin/i386/native_threads/Java -Djava.endorsed.dirs=/usr/local/tomcat-smartask/bin:/usr/local/tomcat-smartask/common/lib - classpath /usr/Java/jdk1.3.1_07/lib/tools.jar:/usr/local/tomcat-smartask/bin/bootstrap.jar -Dcatalina.base=/usr/local/tomcat-smartask -Dcatalina.home=/usr/local/tomcat-smartask org.apache.catalina.startup.Bootstrap start

root 18936 0.0 6.7 236880 86704 ? S Apr28 0:00
/usr/Java/jdk1.3.1_07/bin/i386/native_threads/Java -Djava.endorsed.dirs=/usr/local/tomcat-smartask/bin:/usr/local/tomcat-smartask/common/lib -classpath /usr/Java/jdk1.3.1_07/lib/tools.jar:/usr/local/tomcat-smartask/bin/bootstrap.jar -Dcatalina.base=/usr/local/tomcat-smartask -Dcatalina.home=/usr/local/tomcat-smartask org.apache.catalina.startup.Bootstrap start

There were about 50 just like the above, and this command took care of all of them at once, once I took the two minutes needed to puzzle it out;

kill -9 `ps -auxwww | grep smart | cut -c10-16`

I'm going to assume that most everyone who made it this far understands the above, and is wondering why it took me to this point in my career before I used such a simple damn command.

My answer? Because I never needed it before, so there.

For the rest of you, who must have a masochistic streak a mile wide somewhere in your makeup, the command above is actually 3 commands, enclosed in backquotes and separated by pipes (|), that output a number (the pid) to the fourth command, the kill -9


ps -auxwww - is part of the Berkeley implementation of the the ps command, found inn /usr/ucb for an Solaris users out there. It gives me the entire process entry, instead of cutting it off after display line. If passes that information to

grep smart - grep looks at all the process entries and filters out all the ones where the word "smart" doesn't appear. I can't use "Java" because I have lots of processes with that word in the entry, and I don't want to kill them all. However, if I could have used "Java" then I could have gotten away with the much simpler and more familiar "ps -ef" command at the beginning, rather than the more cumbersome "ps -auxwww". The filtered processes are passed to

cut -c10-16 - which cuts out the characters in the 10th thru 16th spaces in the filtered process entry and serves them to

kill -9 - end a process, and do it now. It is the UNIX equivalent of your mother calling you by your entire name, and telling you to drop whatever you are doing and get in the house now.

I copied the entire command to a text file so that rather than having to type multiple kill -9's in future, I can just change "smart" to whatever identifies the processes I want and run it again. Two minutes expended now, 30 minutes saved over the course of the next few months. The entire course of a system administration career can be traced in minutes of activity saved. Like I said, lazy

The best sysadmins, the ones whose praises are sung in story and song, are entirely sessile, or invisible, having saved enough time that they need never come to work, which of course is the goal of every sysadmin.

Just think how much blogging I'll be able to do then.

Update: Aside from the sloth, another nice thing about thing a sysadmin is that there are always more elegant sysadmins around to point out inefficiencies in one's code**, like Jeff of Caerdroia, who wrote to point out that

kill -9 `ps -auxwww | grep smart | grep -v grep | awk '{print $2}'`

is a much better solution, as it proscribes the possibility (admittedly remote) that I would kill my own command before it had run its course. After all, "smart" would be found in its process entry as well. That's where the grep -v grep command comes in, as it filters out the word "grep." awk '{print $2}' pulls out everything in the second field of the process entry, rather than a set number of characters, useful in case the pids that need killing Re not all of the same length. As my "cut" would only pull out spaces in the case of a shorter pid and extra spaces are ignored in commands, it doesn't actually matter, but it is a more elegant solution, and thus better.

*The sysadmin at Medfusion is, though, and both he and his boss read the site regularly. Heh.

** all it takes is a willingness to expose one's shortcomings to the world, and I posses both the willingness and the shortcomings in ample measure.

Posted by Bigwig at April 30, 2003 03:08 PM | TrackBack
First time visitor to House Hraka? Wondering if everything we produce could possibly be as brilliant/stupid/evil/pedantic/insipid/inspired as the post you just read? Check out the Hraka Essentials, the (mostly) reader-selected guide to Hraka's best posts, and decide for yourself.
Post a comment Note: Comments with more than two dashes per line will be blocked as spam.

Remember personal info?