6 Ways to Improve your IT Documentation

I came into this position last year, knowing that I had a challenge ahead of me.  The previous system administrator left our department with little to no documentation as part of their legacy.  The documentation that was left had not been updated for over six years. You know how that goes with technology, a lot happens over six years. This was a problem, as our server environment had grown over 300% since the introduction of VMware into our environment.   We now had well over 80 servers, most of them new in the past couple years.  No system configuration info, hostnames, IP addresses, purpose/role of the server, and no reason why certain servers had the configurations and setups they did.  Talk about starting from the ground up!  I had to go through each server and document what each one did, where it was in the rack, physical or virtual, who accesses it, is it secured and how so, when does the software maintenance expire, is it backed up and where, I could go for days on what needed to be done and what had not been done!

From my experiences, I have decided to share six items that I recommend to use to help improve documentation in any IT environment:

1. MediaWiki - I setup a WAMP server a few years back and installed MediaWiki on it to use it as a knowledge base.  Doing so helped offload the enormous amount of questions that our field technicians would ask me about. When I moved into a server position, it turned out to be a great tool to use for active documentation as well.  The hard part was getting our techs and other staff to start using it.  Since we all are in the bad habit of not documenting ANYTHING.  It has taken some time, but we are slowly getting everyone on board.  Implementing MediaWiki has helped increase turn around time, system up time, and the need of continuity within department.
   
   
2. Evernote - Anything to take notes with really... Using Evernote during project planning, server installation, migration/upgrade planning, and keeping an up-to-date to-do list, is a quick and easy way to make notes, store online resources, email correspondence, screen clippings, and do it all on the go or from your primary PC.  I use it to take notes during installs, trainings, webinars, and meetings.  It also comes in handy when I can't sleep because of my brain not being able to dump my daily anxiety and the knowledge of tomorrows needs, I can make a quick note on what's bothering me and have it ready to add to my schedule the next day.  Once I have a note completed if it is pertinent to our environment I can copy it into our Wiki Knowledge Base to keep my system documentation updated.
   
   
3. Benefit from Vendor Documentation and Forums - I can't thank EMC enough for how well documented a lot of their products are, most accessible in PDF format.  I'm able to download the doc's, store them on our file server or upload them to our Wiki for later reference.  Thus, increasing our local documentation.  If I don't want to make a call to support and waste their time and mine, I'll take a look through the Admin Guide or Best Practices for certain products, or I'll jump on the forums for the company to see if anyone else has the same issue.  Being in a PDF format is also great if I am looking for a bit of "light" reading, I can upload it to my e-reader, which I have with me 98% of the time.
   
   
4. Blog IT - I don't know how many times I've researched, researched, and Googled errors and had found no results.  I do find one thing in common, other people with the same problem and no answers.  After another day or so of troubleshooting, I finally resolve the issue, then, share it with no one by my Wiki.  This is one reason why I too have started a blog.  I have found so many answers from other administrators, frustrated users, and helpful support personnel on their blogs than from dedicated support sites. Start up a blog and get your time intensive problem resolutions posted so others can benefit from your research.
   
   
5. Take the Extra Time - It's worth it in the end! We all know how much faster it is when you just want to get something quick up and running or fixed, then throw our hands in the air like you just roped a calf in a rodeo event and call it good, without documenting anything. This is what gets us in trouble and how we start to fail at documentation.  Try to make it your best practice to do screenshots, write down steps taken, resources used, contacts who assisted you, and the involvement of others.  Simply taking a few notes during an install,  where you installed it, configurations you made, why you made the change, and keeping that documented in a location accessible to your team can relieve future headaches.  It is never pleasant when you are completed with a project and you don't know where something was left out.  Having the documentation in place and notes taken, benefit you when you are troubleshooting an issue.
   
   
6. Keep Everything  - Cover yourself! You never know when something can come back and bite you.  I keep everything, email, online chats from support, webex recordings from support calls (if you ask nice, sometimes they are more than willing to get you a copy), voice mails and soon, live voice recordings from phone calls.  The way things have turned over the past decade, you need to cover yourself, your family, your coworkers, and your company.  I don't know how many times someone has asked me, "Do you remember what we did three years ago for this person?". We've also had, "I've been waiting for weeks and now I am going up the ladder to complain!" It most times comes down to doing a search in my Outlook, through my PSTs (yes i still use them) or EmailXtender Archive, and there it is an email chain with the person and the proof that I need to back me up.  However, there are downsides to keeping everything. Being the Exchange and EmailXtender administrator, I've had instances where someone requests an investigation to see if a correspondence between two individuals did exist, we can tell if the email has been read, forwarded or replied to very easily.  So in that persons case, the idea of keeping everything unfortunately bit them.
   

You never know what will happen in the future.  For the benefit of your coworkers, your company, your integrity, your legacy, document, document, document!  Please, don't be the conceded, power hungry, arrogant, know-it-all admin that loves watching others fail without your help.  You are hired to help others and provide a service to your company and customers.  Don't leave them hanging if a server rack falls on you.

WSUS Content Folder Filled Up Disk

Today our WSUS server wanted be a pain and inform us that its content folder has reached 120 GB.

After running the Server Cleanup Wizard from the MMC, we found that we needed to have more space cleaned up.

The server is a Virtual Machine, so adding additional space and extending the partition is always easy.  However, we want to avoid continually adding space and extending the partition since we can use the valuable SAN space for more important services.

We are able to clean it up by stopping the WSUS service, delete all the folders and their contents from the content directory. Then from command prompt C:\Program Files\Update Services\Tools, we ran "wsusutil.exe reset".

This allowed the server to then re-download the content and only get what it should need.  This can take some time since there is an enormous amount of updates out there to get for our over 1,500 multi-OS systems.  The last time we did this was about 8 months ago, so it helped for quite a while before needing to be run again.  I will have to do some investigating to see if anyone has a script out there to perform a cleanup every few months.

From here I will monitor the server and if it continues to grow unreasonably, I will probably fire up a 2008 R2 box and start new so we get it off of the 2003 box anyway.

Granting Service Accounts Receive As and Send As Permissions

We recently needed to add a service account to have Send-As and Receive-As permissions on one of our Exchange 2007 databases.  I was able to do so by issuing the following commands.

First, I verified the user did not have the rights about to be assigned.

Get-Mailboxdatabase -Identity DBIdentity | Get-ADPermission -User SERVICEACCOUNT

I did not receive any results from the above cmdlet.

Next, I added the permissions to the database for the service account.


Get-Mailboxdatabase -Identity DBIdentity | Add-ADPermission -User SERVICEACCOUNT -AccessRights ExtendedRight -ExtendedRights receive-as, send-as

Once I issued that command, I verified they applied successfully by running the get-adpermission command again.

This worked for me but I cannot guarantee it will work for others.