Wednesday, May 22, 2013

omreport : failed to load external entity "/opt/dell/srvadmin/var/lib/openmanage/xslroot//oma/cli/about.xsl"

If you're having the following error when executing omreport :
 I/O warning : failed to load external entity "/opt/dell/srvadmin/var/lib/openmanage/xslroot//oma/cli/about.xsl"  
 error  
 xsltParseStylesheetFile : cannot parse /opt/dell/srvadmin/var/lib/openmanage/xslroot//oma/cli/about.xsl  
 Error! XML Transformation failed  

Then install srvadmin-omcommon package :
 # yum install srvadmin-omcommon  

Tuesday, May 21, 2013

DRAC Firmware update failed : Error: 30001 Method httpCgiErrorPage()

Have tried to update an old DRAC4 Firmware from firmware 1.5 to 1.75 via Linux binary and came to an unplaisant surprise :
 Dell Remote Access Controller 4/P  
 The version of this Update Package is newer than the currently installed version.  
 Software application name: Dell Remote Access Controller 4/P Firmware  
 Package version: 1.75  
 Installed version: 1.50  
 Continue? Y/N:Y  
 Executing update...  
 WARNING: DO NOT STOP THIS PROCESS OR INSTALL OTHER DELL PRODUCTS WHILE UPDATE IS IN PROGRESS.  
 THESE ACTIONS MAY CAUSE YOUR SYSTEM TO BECOME UNSTABLE!  
 ......................................................................................
 /tmp/duptmp.xml:6: parser error : Extra content at the end of the document  
 <SVMExecution lang = "en">  
 ^  
 /tmp/.dellSP-XmlResult12908-32487.M19124:6: parser error : Extra content at the end of the document  
 <SVMExecution lang = "en">  
 ^  
 unable to parse /tmp/.dellSP-XmlResult12908-32487.M19124  
 /tmp/.dellSP-XmlResult12908-32487.M19124:6: parser error : Extra content at the end of the document  
 <SVMExecution lang = "en">  
 ^  
 unable to parse /tmp/.dellSP-XmlResult12908-32487.M19124  

Doesn't look good and of course if I try to access the DRAC via HTTPs, I've got a nice CGI error :
 Error: 30001 Method httpCgiErrorPage()  

Looked on the web and somebody (who contacted Dell Support) advises to shutdown the server, unplug the DRAC card for a while and plug it in back... Well explain to your CTO that you need to shutdown a production server, unrack it, unplug a card just because a DRAC update failed o_O
Reference: http://lists.us.dell.com/pipermail/linux-poweredge/2008-January/034556.html

The solution that worked for me was to install the racadm Dell tool on my bastion and reset the firmware remotely.

  • First install racadm :
 # yum install srvadmin-racadm4.x86_64  
Note : This is for DRAC4, didn't had the issue with newer DRAC.
Note 2 : You need to have the Dell OSMA repository installed on your server:
http://www.openfusion.net/linux/dell_omsa

  •  Then run the following command :
 # racadm -rDRAC_IP -i racreset  
Note : Change DRAC_IP with your DRAC IP.
Note 2 : This operation will NOT erase your DRAC configuration.
  •  Wait a while, pray, and if you're lucky as me you should be back on line (with the original firmware version of course).
Final word, I stopped being lazy and updated the firmware via the Web GUI which is a long and annoying process. Of course I used Internet Explorer as I felt like Murphy's law was around this day ^^

Hope that helps !

Yum stuck/hangs at "Running Transaction Test"

If yum is stuck at the "Running Transaction Test" step, double check that you don't have a stalled network mount (NFS,SMB,etc) somewhere.

Umount it and retry your yum/rpm command.

More info on how to umount a stalled NFS share :
http://sysnet-adventures.blogspot.fr/2013/05/umount-stalledfrozen-nfs-mount-point.html

Umount a stalled/frozen NFS mount point

NFS is known to be a little nasty when it comes to umount stalled shares.

Most of the time a simple umount doesn't work, which is a bit frustrating specially when it comes to production servers; The process just hangs and there is no way to interrupt...

Below two procedures to umount stalled  NFS shares. You should try method one before method two as it requires some network "hacks".

Method 1 :

Use a forced lazy umount, this method works 90% of the time :
 # umount -f -l /mnt/nfs_share
Note : Don't use bash auto-completion !!!  


Method 2:

This method is to be used only if method one failed.

The trick is to temporarily steal the NFS server IP adress on the NFS client (the one with stalled mount) so this client thinks that the NFS server is still alive.

Warning : Use method 1 above if your NFS server is still reachable from the NFS client. Otherwise you'll have an IP conflit and trust me you really don't want that to happen.

Let's assume the NFS server IP is 192.168.0.1
  1. Double check that the NFS server is down with ping or nmap.
  2. If your NFS client has very restrictive IPTables rules shut them down temporarily
  3. On the NFS client, set the NFS server IP as secondary address
  4.  # ifconfig eth0:0 192.168.0.1  
     Note : Adjust interface to your own needs  
    
  5. Umount the NFS with a forced lzay umount
  6.  # umount -f -l /mnt/nfs_share  
     Note : Don't use bash auto-completion !!! 
    
  7.  Check that the NFS mount is gone
  8. Remove secondary interface
     # ifconfig eth0:0 down  
     Note : Adjust interface to your own needs  
  9. Restart IPTables if needed
  10. Be happy
  11. Go to sleep, it's been a long day (or night)
 If you have multiple NFS clients stalled, you can set the secondary IP on one client only.
  • Client 1 : Step 1 to 5
  • Client 2 to n : Step 4 and 5
  • Client 1 : Step 6 to 9

This will only work if your NFS client can communicate between each others (watch for IPTables or any other filtering softwares/devices).

Hope that helps ! (that helped me a lot :)

Monday, May 20, 2013

Remove absolute path from MD5 file

The following command will remove absolute paths from a md5 file :
  sed -i -r "s/ .*\/(.+)/  \1/g" file.md5  
This is quite useful when you download backup files which have MD5 containing absolut paths.

For example :
 $ cat file.md5  
 8ee6e966f2cb7a84620ed137604e00c5 /data/prod/file  

If you want to check this file on another server, you won't be able to do it unless you put the data file in the exact same directory (/data/prod/).

After running the above sed command, your md5 file will look like :
 $ cat file.md5  
 8ee6e966f2cb7a84620ed137604e00c5 file  

You can then check your md5 print with "md5sum -c file.md5". You just need to have the MD5 file in the same directory as the data file.

Whitelist files with Clamav

Exclude files from scan :

Sometimes, you will need to whitelist files from a scan, clamscan offers the --exclude option but its usage is not really user-friendly...

Imagine that you need to exclude two files /data/rep/file1 and /data/rep2/file2, the command line would be :
 # clamscan -r -i --exclude=/data/rep/file1 --exclude=/data/rep2/file2  

This is fine if you have few files to whitelist but it quickly becomes unreadable when you have dozen files and directories.

The solution is to input a file to clamscan with xargs. Create a text file containing all files/directories you need to whiltelist (one file/directory per line) :
 # cat /var/lib/clamav/whitelist-files.txt  
 /data/rep/file1  
 /data/rep2/file2  

You can also add regexp like *.mp3 (be aware that this is quite dangerous)

Run clamscan with the following command :
 # sed -e 's/^/--exclude=/' /var/lib/clamav/whitelist-files.txt | xargs clamscan -r -i /directory_to_scan/  

Don't forget to put double quotes or escape when you exclude paths with special characters (especially spaces).

Last but not least, always double check that the files you're whitelisting are completely safe. You can check that out with a meta AV engine like Jotti :
http://virusscan.jotti.org/en

Whitelist a virus signature :

To whitelist a virus a signature, you need to get the ClamAV signature definition, this is the code you have on the right side of the infected file line. For example :
 /data/file.flv: CVE_2012_0773-2 FOUND  

In this case the signature definition is CVE_2012_0773-2, add it to /var/lib/clamav/whitelist-signatures.ign2

That's all ! Be very cautious when whitelisting Virus signatures.
Hope that helps !

Tuesday, May 7, 2013

Enable LDAP SSL/TLS user authentification in Zabbix

By default Zabbix Web interface doesn't offer the SSL/TLS encryption option for the LDAP connector however the feature is available in the PHP code.

If you need to enable the startTLS feature, you will have to edit the PHP file manually :

Edit /usr/share/zabbix/include/classes/class.cldap.php on Zabbix Web server :
 vi /usr/share/zabbix/include/classes/class.cldap.php  

Search for the 'starttls' definition (line 44) and set the value to 'true' :
 'starttls' => true,  

Save the file and you're good to go !

Hope that helps !