Manage your backup retentions policies with retdo

You have folders with all your backups but storage capacity is starting to become low ? Need to clean up all these old files but still want to keep some of them just in case ? Then retdo is the perfect tool !

Retdo is a little script I wrote that allows administrators to clean up files on a custom retention basics.
Retdo can be used to implement production's backups retention plans.

retdo can resolve the following queries :

- I want to keep only one file per week if files are older than 3 months up to 6 months.
- I want to keep only one file per month if files are older than 6 months up to 1 year.
- I want files older than 1 year to be moved to another machine.
- I want a cup of tea (feature in progress)

Code and instructions are available for free at https://github.com/gcharot/retdo

Example : 

 Let's say I have my January daily backups in /data/backup/db/dbname :

#  ll /data/backup/db/dbname/   
 -rw-r--r-- 1 root root 0 Jan 1 12:00 jan01.tgz  
 -rw-r--r-- 1 root root 0 Jan 2 12:00 jan02.tgz  
 -rw-r--r-- 1 root root 0 Jan 3 12:00 jan03.tgz  
 -rw-r--r-- 1 root root 0 Jan 4 12:00 jan04.tgz  
 -rw-r--r-- 1 root root 0 Jan 5 12:00 jan05.tgz  
 -rw-r--r-- 1 root root 0 Jan 6 12:00 jan06.tgz  
 -rw-r--r-- 1 root root 0 Jan 7 12:00 jan07.tgz  
 -rw-r--r-- 1 root root 0 Jan 8 12:00 jan08.tgz  
 -rw-r--r-- 1 root root 0 Jan 9 12:00 jan09.tgz  
 -rw-r--r-- 1 root root 0 Jan 10 12:00 jan10.tgz  
 -rw-r--r-- 1 root root 0 Jan 11 12:00 jan11.tgz  
 -rw-r--r-- 1 root root 0 Jan 12 12:00 jan12.tgz  
 -rw-r--r-- 1 root root 0 Jan 13 12:00 jan13.tgz  
 -rw-r--r-- 1 root root 0 Jan 14 12:00 jan14.tgz  
 -rw-r--r-- 1 root root 0 Jan 15 12:00 jan15.tgz  
 -rw-r--r-- 1 root root 0 Jan 16 12:00 jan16.tgz  
 -rw-r--r-- 1 root root 0 Jan 17 12:00 jan17.tgz  
 -rw-r--r-- 1 root root 0 Jan 18 12:00 jan18.tgz  
 -rw-r--r-- 1 root root 0 Jan 19 12:00 jan19.tgz  
 -rw-r--r-- 1 root root 0 Jan 20 12:00 jan20.tgz  
 -rw-r--r-- 1 root root 0 Jan 21 12:00 jan21.tgz  
 -rw-r--r-- 1 root root 0 Jan 22 12:00 jan22.tgz  
 -rw-r--r-- 1 root root 0 Jan 23 12:00 jan23.tgz  
 -rw-r--r-- 1 root root 0 Jan 24 12:00 jan24.tgz  
 -rw-r--r-- 1 root root 0 Jan 25 12:00 jan25.tgz  
 -rw-r--r-- 1 root root 0 Jan 26 12:00 jan26.tgz  
 -rw-r--r-- 1 root root 0 Jan 27 12:00 jan27.tgz  
 -rw-r--r-- 1 root root 0 Jan 28 12:00 jan28.tgz  
 -rw-r--r-- 1 root root 0 Jan 29 12:00 jan29.tgz  
 -rw-r--r-- 1 root root 0 Jan 30 12:00 jan30.tgz  
 -rw-r--r-- 1 root root 0 Jan 31 12:00 jan31.tgz  

Now I need to free some space up so I'd like to keep only one file per week :

 # retdo -p /data/backup/db/dbname -r "*.tgz" -b 1 -e 92 -d 7  
 26 file(s) processed - 0 file(s) in error  
 # ll /data/backup/db/dbname  
 total 0  
 -rw-r--r-- 1 root root 0 Jan 5 12:00 jan05.tgz  
 -rw-r--r-- 1 root root 0 Jan 12 12:00 jan12.tgz  
 -rw-r--r-- 1 root root 0 Jan 19 12:00 jan19.tgz  
 -rw-r--r-- 1 root root 0 Jan 26 12:00 jan26.tgz  
 -rw-r--r-- 1 root root 0 Jan 31 12:00 jan31.tgz  

As you can see only one file per week (7 days) has been kept, 26 files were deleted.

This commands means  : "find all files matching regexp *.tgz in /data/backup/db/dbname which are older than 1 days up to 92 days (3 months) and keep only one file every week (7 days)"

Hope that helps !

Postfix : Show next deferred delivery attempt

Deferred mails happen, that's the hard life of the Internet. If you want to know when these mails will be requeued, you need to look at the Postfix's spool directory files.

First of all, get the mail's queue ID with the mailq, postqueue commands or the maillog file.
Once you have your queue ID type the following command :

 # find /var/spool/postfix/deferred/ -name QUEUE_ID -exec stat {} \;  

This will find your mail in the deferred spool and show its file's properties.
Postfix stamps mail's files with an access time in the future. This time is the time Postfix will requeue the mail for delivery.

Below a concrete example with the queue ID  3EC905800CB :

 # date  
 Mon Feb 24 18:32:27 CET 2014  
 # find /var/spool/postfix/deferred/ -name 7A5F2580101 -exec stat {} \;  
  File: `/var/spool/postfix/deferred/7/7A5F2580101'  
  Size: 28779      Blocks: 64     IO Block: 4096  regular file  
 Device: 811h/2065d   Inode: 5767425   Links: 1  
 Access: (0700/-rwx------) Uid: (  89/ postfix)  Gid: (  89/ postfix)  
 Access: 2014-02-24 19:22:30.000000000 +0100  
 Modify: 2014-02-24 19:22:30.000000000 +0100  
 Change: 2014-02-24 18:15:50.156029044 +0100  

 
As you can see the access/modify time is a time in the future, that means postfix will requeue the mail at 19:22:30.

Note : Requeuing doesn't mean that the mail will be sent at this exact time, it will depend on your active queue load.

Note 2: You can force a requeuing with the "postsuper -r" command

Smartctl : Linux disk I/O scheduler is reseted back to default's CFQ

Got a weird issue recently, I'm monitoring my SSD's life time with smartctl + Zabbix and realized that my scheduler settings are reseted each time smartctl was executed !

 # echo noop > /sys/block/sda/queue/scheduler  
   
 # cat /sys/block/sda/queue/scheduler  
 [noop] anticipatory deadline cfq  
   
 # smartctl -A --device=sat+megaraid,0 /dev/sda  
 smartctl 5.43 2012-06-30 r3573 [x86_64-linux-2.6.32-358.23.2.el6.x86_64] (local build)  
 Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net  
 === START OF READ SMART DATA SECTION ===  
 ...  
 ...  
   
 # cat /sys/block/sda/queue/scheduler  
 noop anticipatory deadline [cfq]  

There is no real solution, but you can work around by specifying  the generic SCSI name i.e "sgX "instead of sdX.

 # echo noop > /sys/block/sda/queue/scheduler  
   
 # cat /sys/block/sda/queue/scheduler  
 [noop] anticipatory deadline cfq  
   
 # smartctl -A --device=sat+megaraid,0 /dev/sg0  
 smartctl 5.43 2012-06-30 r3573 [x86_64-linux-2.6.32-358.23.2.el6.x86_64] (local build)  
 Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net  
 === START OF READ SMART DATA SECTION ===  
 ...  
 ...  
   
 # cat /sys/block/sda/queue/scheduler  
  [noop] anticipatory deadline cfq 

And voila ! Problem not really solved but that does the job !

You can use sg_map (part of the sg3_utils package) to check the sdX -> sgX mappings :

 # sg_map -a  
 /dev/sg0 /dev/sda  
 /dev/sg1 /dev/sdb  
 /dev/sg2 /dev/scd0  

Omreport fails : object not found

If you get the following message while using omreport :

 $ omreport chassis memory  
 Memory Information  
 Error : Memory object not found  
 $ omreport chassis hwperformance  
 Error! No Hardware Peformance probes found on this system.  

The first thing to do is to restart the srvadmin services :

 # srvadmin-services.sh restart  
 # service ipmi restart  

Check that the services are properly started.

If that doesn't solve the problem, you might have a semaphore issue. In my case Zabbix agent/scripts became nuts and didn't close its semaphores.

To list the current semaphore's arrays use the following command :

 # ipcs -s  

To show the current system limits

 # ipcs -sl  

You can use the following command to count the current number of semaphore's arrays

 # ipcs -us  

If you reached the system limit, it will certainly explain the omreport issue. From now on, you have two possibilities :

• You've reached the limit because there is an issue on your system (semaphores not closed or whatever reason). You need to cleanup your semaphores with the following command :

 # ipcrm -s semaphore_id  
 To clean all semaphores from a particular user :  
 # ipcs -s | awk '/username/ {system("ipcrm -s" $2)}'   

Important : You need to stop attached process before removing the semaphores.

• All your semaphores are legit, you need to increase the system limits :

https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Tuning_and_Optimizing_Red_Hat_Enterprise_Linux_for_Oracle_9i_and_10g_Databases/sect-Oracle_9i_and_10g_Tuning_Guide-Setting_Semaphores-Setting_Semaphore_Parameters.html

Hope that helps !

Easy way to find your public IP in scripts or CLI

There is a lot of online tools that gives your public IP but most of them are either not accepting cli User-Agent or they require nasty parsing with AWK,grep,sed,etc

Best way to get your public IP address is to use ifconfig.me which only returns the IP; simple and efficient.

 $ curl http://ifconfig.me  

The only problem is the responsiveness (few seconds).

Another cool way is to query the opendns servers :

 $ dig +short myip.opendns.com @resolver1.opendns.com

This method is pretty fast but requires the dig command (part of the bind-utils package).

Dell Firmware update fails with "mktemp: too many templates"

If you have the following error while updating a Dell server firmware (BIOS, RAID, etc) via Linux binary (*.BIN) :

 mktemp: too many templates  

Then check the binary's filename for specials characters, in my case Chrome added a "(1)" at the end of the filename. Remove it, restart the update process and you're good to go !

Bash : Wait for a command with timeout

Here is a very useful little command that wait for a process to finish and kill it if doesn't exit after a pre defined timeout.

The command is called timeout and is part of the coreutils package witch is embedded by default in CentOS 6.

Usage is very simple, you just need to specify the timeout, the process to start and optionally the signal to send in case of failure.

Timeout returns the exit value of the process and if the timeout has been reached, the value 124 is returned.

Here are some examples on how to use timeout :

Successful process : 

 $ timeout 10s ls /tmp  
 ....  
 ....  
 $ echo $?  
 0  

Unsuccessful process :

 $ timeout 10s ls /blabla  
 ls: cannot access /blabla: No such file or directory  
 $ echo $?  
 2  

Timed out process :

 $ timeout 5s sleep 10  
 $ echo $?  
 124  

More details in the man page.

Hope that helps !

Cobbler reposync failed

I run daily cobbler reposync crons and it appears sometimes the process fails with the following error :

 Exception occured: <class 'cobbler.cexceptions.CX'>  
 Exception value: 'cobbler reposync failed'  
 Exception Info:  
  File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 126, in die  
   raise CX(msg)  
   
 Exception occured: <class 'cobbler.cexceptions.CX'>  
 Exception value: 'cobbler reposync failed'  
 Exception Info:  
  File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line 125, in run  
   self.sync(repo)  
   File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line 169, in sync  
   return self.yum_sync(repo)  
   File "/usr/lib/python2.6/site-packages/cobbler/action_reposync.py", line 402, in yum_sync  
   utils.die(self.logger,"cobbler reposync failed")  
   File "/usr/lib/python2.6/site-packages/cobbler/utils.py", line 134, in die  
   raise CX(msg)  
...
!!! TASK FAILED !!!

I don't really have an explanation for this but it seems that the reposync process doesn't really fail as if I run the reposync command manually the process goes fine.

The createrepo command line is shown when you execute "cobbler reposync", for example :

 hello, reposync  
 run, reposync, run!  
 creating: /var/www/cobbler/repo_mirror/Dell-CentOS5/.origin/Dell-CentOS5.repo  
 running: /usr/bin/reposync -l -m -d --config=/var/www/cobbler/repo_mirror/Dell-CentOS5/.origin/Dell-CentOS5.repo --repoid=Dell-CentOS5 --download_path=/var/www/cobbler/repo_mirror  

In this case the command line is :

reposync -l -m -d --config=/var/www/cobbler/repo_mirror/Dell-CentOS5/.origin/Dell-CentOS5.repo --repoid=Dell-CentOS5 --download_path=/var/www/cobbler/repo_mirror  

The process should go fine, however the return value $? is '1' which can explain why the cobbler commands fails.

Emulate bad or WAN network performances from a particular IP on a Gigabit LAN network

If you're developing Web or mobile applications, you'll certainly be confronted to poor network conditions.

The problem now is "how can I test my application under bad network conditions". Well you could rent a forein internet connection or use tools that reports performance from various remote countries however this is not a good debugging environment.

The solution is to use TC and NetEM on your front development server (typically Web or reverse proxy server), then use filters so only one client station (the debugging station) is impacted.
Don't forget to use filter otherwise all your clients will be impacted.

Below an example on how to emulate a network with :

• 1Mbps bandwidth
• 400ms delay
• 5% packet loss
• 1% Corrupted packet
• 1% Duplicate packet

The debugging client IP is 192.168.0.42  (i.e the IP impacted by the bad network performance);
The following commands need to be executed on the front developement server, please set the appropriate NIC for you environment (eth0 used below) :

 # Clean up rules  
   
 tc qdisc del dev eth0 root  
   
 # root htb init 1:  
   
 tc qdisc add dev eth0 handle 1: root htb  
   
 # Create class 1:42 with 1Mbps bandwidth  
   
 tc class add dev eth0 parent 1:1 classid 1:42 htb rate 1Mbps  
   
 # Set network degradations on class 1:42  
   
 tc qdisc add dev eth0 parent 1:42 handle 30: netem loss 5% delay 400ms duplicate 1% corrupt 1%  
   
 # Filter class 1:42 to 192.168.0.42 only (match destination IP)  
   
 tc filter add dev eth0 protocol ip prio 1 u32 match ip dst 192.168.0.42 flowid 1:42  
   
 # Filter class 1:42 to 192.168.0.42 only (match source IP)  
   
 tc filter add dev eth0 protocol ip prio 1 u32 match ip src 192.168.0.42 flowid 1:42  

To check that the rules are properly set use the following commands :

 tc qdisc show dev eth0  
 tc class show dev eth0  
 tc filter show dev eth0  

Once you're done with the testing, cleanup the rules with the command :

 tc qdisc del dev eth0 root   

There is many other options you can use (correlation, distribution, packet reordering, etc), please check the documentation available at :

http://www.linuxfoundation.org/collaborate/workgroups/networking/netem

If this setup fits your requirements, I advice you to create a shell script so you can start/stop the rules with custom values. Be aware that you can also make filters based on source/destination ports, etc.

If you have more complex requirements, you can try WANem, which is a live Linux Distribution with a graphical interface on top of NetEM. Please be aware that this requires route modifications on your client and server (or any other routing tricks).

http://wanem.sourceforge.net/
http://sourceforge.net/projects/wanem/files/Documents/WANemv11-Setup-Guide.pdf

I didn't had the opportunity to try it, please let me know if you have any feedback.

DRAC Firmware update failed : Error: 30001 Method httpCgiErrorPage()

Have tried to update an old DRAC4 Firmware from firmware 1.5 to 1.75 via Linux binary and came to an unplaisant surprise :

 Dell Remote Access Controller 4/P  
 The version of this Update Package is newer than the currently installed version.  
 Software application name: Dell Remote Access Controller 4/P Firmware  
 Package version: 1.75  
 Installed version: 1.50  
 Continue? Y/N:Y  
 Executing update...  
 WARNING: DO NOT STOP THIS PROCESS OR INSTALL OTHER DELL PRODUCTS WHILE UPDATE IS IN PROGRESS.  
 THESE ACTIONS MAY CAUSE YOUR SYSTEM TO BECOME UNSTABLE!  
 ......................................................................................
 /tmp/duptmp.xml:6: parser error : Extra content at the end of the document  
 <SVMExecution lang = "en">  
 ^  
 /tmp/.dellSP-XmlResult12908-32487.M19124:6: parser error : Extra content at the end of the document  
 <SVMExecution lang = "en">  
 ^  
 unable to parse /tmp/.dellSP-XmlResult12908-32487.M19124  
 /tmp/.dellSP-XmlResult12908-32487.M19124:6: parser error : Extra content at the end of the document  
 <SVMExecution lang = "en">  
 ^  
 unable to parse /tmp/.dellSP-XmlResult12908-32487.M19124  

Doesn't look good and of course if I try to access the DRAC via HTTPs, I've got a nice CGI error :

 Error: 30001 Method httpCgiErrorPage()  

Looked on the web and somebody (who contacted Dell Support) advises to shutdown the server, unplug the DRAC card for a while and plug it in back... Well explain to your CTO that you need to shutdown a production server, unrack it, unplug a card just because a DRAC update failed o_O
Reference: http://lists.us.dell.com/pipermail/linux-poweredge/2008-January/034556.html

The solution that worked for me was to install the racadm Dell tool on my bastion and reset the firmware remotely.

• First install racadm :

 # yum install srvadmin-racadm4.x86_64  

Note : This is for DRAC4, didn't had the issue with newer DRAC.
Note 2 : You need to have the Dell OSMA repository installed on your server:
http://www.openfusion.net/linux/dell_omsa

•  Then run the following command :

 # racadm -rDRAC_IP -i racreset  

Note : Change DRAC_IP with your DRAC IP.
Note 2 : This operation will NOT erase your DRAC configuration.

•  Wait a while, pray, and if you're lucky as me you should be back on line (with the original firmware version of course).

Final word, I stopped being lazy and updated the firmware via the Web GUI which is a long and annoying process. Of course I used Internet Explorer as I felt like Murphy's law was around this day ^^

Hope that helps !

Yum stuck/hangs at "Running Transaction Test"

If yum is stuck at the "Running Transaction Test" step, double check that you don't have a stalled network mount (NFS,SMB,etc) somewhere.

Umount it and retry your yum/rpm command.

More info on how to umount a stalled NFS share :
http://sysnet-adventures.blogspot.fr/2013/05/umount-stalledfrozen-nfs-mount-point.html

Umount a stalled/frozen NFS mount point

NFS is known to be a little nasty when it comes to umount stalled shares.

Most of the time a simple umount doesn't work, which is a bit frustrating specially when it comes to production servers; The process just hangs and there is no way to interrupt...

Below two procedures to umount stalled  NFS shares. You should try method one before method two as it requires some network "hacks".

Method 1 :

Use a forced lazy umount, this method works 90% of the time :

 # umount -f -l /mnt/nfs_share
Note : Don't use bash auto-completion !!!  

Method 2:

This method is to be used only if method one failed.

The trick is to temporarily steal the NFS server IP adress on the NFS client (the one with stalled mount) so this client thinks that the NFS server is still alive.

Warning : Use method 1 above if your NFS server is still reachable from the NFS client. Otherwise you'll have an IP conflit and trust me you really don't want that to happen.

Let's assume the NFS server IP is 192.168.0.1

1. Double check that the NFS server is down with ping or nmap.
2. If your NFS client has very restrictive IPTables rules shut them down temporarily
3. On the NFS client, set the NFS server IP as secondary address

 # ifconfig eth0:0 192.168.0.1  
 Note : Adjust interface to your own needs  

4. Umount the NFS with a forced lzay umount

 # umount -f -l /mnt/nfs_share  
 Note : Don't use bash auto-completion !!! 

5.  Check that the NFS mount is gone
6. Remove secondary interface
   

    # ifconfig eth0:0 down  
    Note : Adjust interface to your own needs  

7. Restart IPTables if needed
8. Be happy
9. Go to sleep, it's been a long day (or night)

 If you have multiple NFS clients stalled, you can set the secondary IP on one client only.

• Client 1 : Step 1 to 5

• Client 2 to n : Step 4 and 5

• Client 1 : Step 6 to 9

This will only work if your NFS client can communicate between each others (watch for IPTables or any other filtering softwares/devices).

Hope that helps ! (that helped me a lot :)

Remove absolute path from MD5 file

The following command will remove absolute paths from a md5 file :

  sed -i -r "s/ .*\/(.+)/  \1/g" file.md5  

This is quite useful when you download backup files which have MD5 containing absolut paths.

For example :

 $ cat file.md5  
 8ee6e966f2cb7a84620ed137604e00c5 /data/prod/file  

If you want to check this file on another server, you won't be able to do it unless you put the data file in the exact same directory (/data/prod/).

After running the above sed command, your md5 file will look like :

 $ cat file.md5  
 8ee6e966f2cb7a84620ed137604e00c5 file  

You can then check your md5 print with "md5sum -c file.md5". You just need to have the MD5 file in the same directory as the data file.

Whitelist files with Clamav

Exclude files from scan :

Sometimes, you will need to whitelist files from a scan, clamscan offers the --exclude option but its usage is not really user-friendly...

Imagine that you need to exclude two files /data/rep/file1 and /data/rep2/file2, the command line would be :

 # clamscan -r -i --exclude=/data/rep/file1 --exclude=/data/rep2/file2  

This is fine if you have few files to whitelist but it quickly becomes unreadable when you have dozen files and directories.

The solution is to input a file to clamscan with xargs. Create a text file containing all files/directories you need to whiltelist (one file/directory per line) :

 # cat /var/lib/clamav/whitelist-files.txt  
 /data/rep/file1  
 /data/rep2/file2  

You can also add regexp like *.mp3 (be aware that this is quite dangerous)

Run clamscan with the following command :

 # sed -e 's/^/--exclude=/' /var/lib/clamav/whitelist-files.txt | xargs clamscan -r -i /directory_to_scan/  

Don't forget to put double quotes or escape when you exclude paths with special characters (especially spaces).

Last but not least, always double check that the files you're whitelisting are completely safe. You can check that out with a meta AV engine like Jotti :
http://virusscan.jotti.org/en

Whitelist a virus signature :

To whitelist a virus a signature, you need to get the ClamAV signature definition, this is the code you have on the right side of the infected file line. For example :

 /data/file.flv: CVE_2012_0773-2 FOUND  

In this case the signature definition is CVE_2012_0773-2, add it to /var/lib/clamav/whitelist-signatures.ign2

That's all ! Be very cautious when whitelisting Virus signatures.
Hope that helps !

Finding less used wireless channel on Linux

Want to find the less used wireless channel around you ? This little one liner will give a summary of all wireless channels with the number of SSID associated.

 echo "Nb SSID - Channel" ; iwlist scan 2>/dev/null | grep "Channel:" | cut -d':' -f2 | sort -n | uniq -c   
 Nb SSID - Channel  
       4 1  
       3 2  
       3 3  
       1 5  
       5 6  
       4 7  
       3 9  
       4 11  
       3 12  

Right column is the Channel number and left column is the number of SSID found on this channel.

 Of course, this is just a basic overview, for real deployments you should use a more sophisticated tool like inSSIDer.

Also an interesting post on how to choose the right channel :
http://www.dslreports.com/faq/14250

Set scale in (non interactive) scripts with the bc command

If you need to do some kind of math operation in shell scripts you might want to write :

 $ echo "9/2" | bc  
 4  

By default bc truncate the result which is bit annonying...

Using bc -l option gives you a far too precise result :

 $ echo "9/2" | bc -l  
 4.50000000000000000000  

Fortunately, bc comes with the "scale" option to set the scale to whatever presision you wish.

To do it non interactively, you need to specify the scale in the echo before the operation.
For example, to set a precision of 2 digits :

 $ echo -e "scale=2 \n 9/2" | bc  
 4.50  

So the script code would look like :

 val=$(echo -e "scale=2 \n 9/2" | bc)  

Replace end of lines (\n) by any character

Just found out a classic UNIX command I never used before.

The command "paste" allows you to replace end of lines (i.e \n) by any character of your choice.

I normally used sed for this purpose but the syntax is quite...dirty. For example to replace all end of lines by a space with sed, the command is :

 $ sed ':a;N;$!ba;s/\n/ /g' /path/to/file  

With paste the syntax is much more human readable :

 $ paste -s /path/to/file  

By default "paste" replaces end of lines with tabs, to specify a delimiter use the -d option.

Replace end of lines with spaces :

 $ paste -s -d' ' /path/to/file  

Replace end of lines with commas :

 $ paste -s -d',' /path/to/file  

 Use '-' for reading from stdin :

 echo -e "a\nb\nc\nd" | paste -s -d',' -  
 a,b,c,d  

More info in the man as always.

This tips will not change your life but i found it quite useful !

Hope that helps !