My Linux Remote Administration Toolkit
Perhaps my most favorite capability afforded by Linux is ease of remote administration. While it is true that Windows administrators have access to a similar function using Terminal Services I have found that it is not the simple and elegant interface offered by Linux. I also like the simple yet powerful tools available for administration from the Linux command line. Here is a review of the tools I use most frequently.
OpenSSH and keychain
OpenSSH is one of several packages that make remote administration of Linux possible however the powerful add on Keychain enhances the power of SSH quite nicely. Keychain works in concert with ssh-agent to maintain a single ssh-agent session across multiple logins. This saves the administrator the hassle of having to enter a passphrase multiple times during a given login session. It also offers the ability to use RSA/DSA keys with cron jobs in an easy and secure way. This brings me to the next tool on my belt; cron.
Cron is a tool that allow administrators, and sometimes users, the ability to schedule tasks, a Windows analog being the "Scheduled Tasks" mechanism. I like to use cron to automate certain tasks such as tripwire scanning and reporting, searching for rootkits and executing log parsing scripts. Using cron one can execute a command or script at 3:51am on the third Tuesday of every month. It can also be configured to perform a task hourly or daily. Cron is my favorite tool for automating tasks.
screen, tail and swatch
Screen is a tool that often makes these "Top 10" lists and for good reason. Screen makes it simple to run multiple sessions on a remote system and it even allows you to detach these session and reconnect later. This is handy for starting a kernel compile or download, detach the session to work on something else, and return later to make sure everything went as planned. I like to use tail and swatch with screen to keep an eye on logs while chasing down a problem with a service. swatch is great for ongoing log monitoring with screen because you can use regex to assign color codes to various events which allows for "at-a-glance" quick analysis.
less, lynx and man
How often has "RTFM" been offered as an answer to a question on a mailing list or in a channel on IRC? Reading documentation is a crucial element of system administration and these three tools make that task simple. Most everyone should already be familiar with man pages. The pager less works well for reading README and INSTALL documents. Some of you may be tilting your head sideways wondering why I would include lynx among my favorite tools for administration. Why not use a modern browser such as Firefox or Konqueror from my workstation? There are two reasons really. First, more documentation bundled with software is included in html format and lynx makes reading this documentation fast and easy. The second reason is that when I am working on a server I like to stay focused on the task at hand. By using lynx to search the web for help I remain focused on the task at hand and resist the temptation to click the Slashdot icon on my toolbar.
sed, awk and grep
Rounding out my favorite system administration tools are sed, awk and grep. Again, I use these tools mainly for quickly analyzing logs or the output from various commands and scripts. I use grep and awk together frequently to quickly grab bits of interesting information from mail logs. If deeper analysis is required I generally throw together a perl script and call it a day. As my knowledge of perl has expanded I find myself using sed less frequently however it is handy for making quick changes to bind zone files or any configuration file where a template can be used.
Wrapping Things Up
The twelve programs mentioned here are the tools that I use most frequently when working on remote servers. I consider these utilities to be among the most useful for keeping a server operating smoothly. Without these tools it would be much more difficult to leverage the power of snort, iptables and server daemons such as apache and bind. Just as a building needs a good foundation a good administrator needs to good foundation of tools upon which he or she can build complex and reliable systems. These tools are my foundation.