Spikes in xorg and Jerky Video in MythTV and VLC….

This might be of some use to folks running Gnome 2.24 in Ubuntu or Fedora or whatever.  I was recently trying to debug a very annoying problem where every minute or so my MythTV playback would get jittery with a corresponding CPU spike in xorg usage.  The same thing would happen in other video players like VLC.  It didn’t make any difference what kind of output rendering was being used.

It turns out this extremely annoying problem is due to stupid gnome-screensaver!  At least it was on my machine.  Take a look at this.  It’s supposed to be fixed in Gnome 2.26+.  Well, I just thought I’d put this one out there, because it’s very difficult to debug, and I see occasional posts about jittery video playback in Linux.  Of course, a great majority of those problems are due to proprietary video drivers though.

Filed under: Uncategorized , gnome, jittery video, mythtv, stuttering video, vlc, xorg cpu spike

VOX Recording in Linux (part 2)

After I made my last post about how to do line-in recordings from your scanner radio with silence detection and cutting, also known as VOX recording in the old days, I started looking at SoX the Linux Sound eXchange CLI audio processing application. Using SoX is a lot slicker than my arecord-mp3splt-mpgjoin method. It lets you do the recording, silence detection, and mp3 encoding in one line, and it puts everything into one audio file, no muss no fuss.

Here are some examples:

rec -r 44100 -p | sox -p OUTPUT.mp3 silence -l 1 00:00:00.5 -45d -1 00:00:00.5 -45d

rec -r 22050 -c 1 -p | sox -p OUTPUT.mp3 silence -l 1 00:00:00.5 -45d -1 00:00:00.5 -45d

The first one will give you a 44.1 kHz, stereo, 128 kbps MP3. The second one will give you a 22.05 kHz, mono, 32 kbps MP3.

The rec command does the recording, -r specifies the sampling rate and -c sets the amount of audio channels, “-p” means the input is piped to another command. The sox pipe also needs a “-p” to tell it that its input is coming from a pipe, in this case the rec command.

The crypt CLI switches in the piped sox command “silence -l 1 00:00:00.5 -45d -1 00:00:00.5 -45d” is what does the silence detection and trimming. It means to cut silence from the beginning of the file and parts within the middle and end. “xx:xx:xx.x” is in hh:mm:ss.s format and tells sox the minimum duration to consider valid audio. In my examples I have this set to 0.5 seconds for valid audio. The “-45d” means -45dB and is the minimum valid audio level to do trimming. In other words, if you were to say -100d, basically everything would be considered valid and there would be no trimming, you will need to play with this depending on your input levels.

I made a test for almost 2 hours with my scanner connected and scanning the aircraft bands. I ended up with a 22 minute file. Very nice!! Now I’d say we have a very nice Linux-style “scanner recorder” program, and it’s a hell of a lot slicker than anything you’d find in Windows.

If you are a Debian or Ubuntu user, you will need to compile your own version of SoX if you want to encode to MP3. This bug explains that the version of SoX in the Debian repos was compiled without MP3 support. Anyway, all you need to do is download the latest source, make sure you have all the build deps (apt-get build-dep sox), make sure you have the libmp3lame and libmp3lame-dev packages installed, then just do ./configure, make, make install.

When you do the ./configure you’ll see a list at the end of the output of what capabilities will be compiled into sox, just make sure that it says you will have MP3 support. Your compiled version will go into /usr/local by default, so you don’t need to uninstall the repo version of sox, since that one is in /usr and programs in /usr/local always take precedance. You can verify that by doing “which sox” after installing it and it should show you “/usr/local/bin/sox”.

Filed under: Audio , arecord, Audio, ham radio, lame, pulseaudio, scanner recorder, silence detection, silence trim, sox, voice operated recording, vox

Voice Operated Recording in Linux (VOX recording)

Many ham radio operators and scanner enthsiasts have a need to do long time recordings from a line-in source (such as a scanner or ham radio), but they don’t want to record all of the silence in between the transmissions.  Enter the concept of VOX recording.  Those of you that used to use tape recorders back in the good ol days will remember the VOX detection circuit that the higher end recorders used to have.  So, how do we do this with a computer?  In Windows there are a number of applications, Scanner Recorder and Rec-All Pro come to mind.  These apps let you record from the line in and pause the recording when the levels go below a user specified threshold.  They also let you record directly to MP3 as well as set the sampling and bit rate, for compression of the recording.

In Linux I’ve been able to get Rec-All Pro to work, but it’s kind of a kludge since you have to use Wine.  It works, but you cannot encode directly to MP3, I assume this is because of the way the Wine programs use DLLs.  I never was able to figure it out.  Anyway, I prefer to use a Linux native method.

I start off by setting up the radio to the station I want to record, then I use the PulseAudio VU meter along with the Alsa Mixer to set my Line-In level and Capture level to the proper amplitude, by watching feedback from the VU meter.  Then I execute the following to begin the recording:

arecord -D plughw:0,0 -f cd | lame -m s -a -b 64 - FILE.mp3
or
arecord -D plughw:0,0 -f S16_LE -c1 -r22050 -t raw | lame -r -s 22.05 -m m -b 64 - FILE.mp3

The first command will produce a file called “FILE.mp3″ with 16 bit resolution, 44.1 kHz sampling, mono. The second command will encode to 16 bit, 22 kHz, mono.

When you are finished recording do a Ctrl-C to end arecord. Now we need to do the silence detection.

mp3splt -s -p th=-30,min=1,rm FILE.mp3

The command above will split the input file using a silence threshold of -30 dB, a minimum silence duration of 1 second, and removes the silence from the resulting split files. And you will end up with a directory full of “Track 0*.mp3″ files. Now we just need to join all these files into one file. So go ahead and execute:

mpgjoin *.mp3 -o JOINED.mp3

Below is a little script to automate all of this. The only thing you need to change before running it is the bit in the arecord command “-d 300″. Change the 300 part to however many seconds you want to run the recording for.

#!/bin/sh
#USAGE: Records from line-in for -d xx seconds, then splits tracks removing silence between tracks, and combines all into one file
arecord -D plughw:0,0 -f S16_LE -c1 -r22050 -t raw -d 300 | lame -r -s 22.05 -m m -b 64 - Recording.mp3 ;
sleep 3 ;
mp3splt -s -p th=-30,min=1,rm Recording.mp3 ;
sleep 2 ;
mv Recording.mp3 Recording.mp3.original ;
sleep 2 ;
mpgjoin *.mp3 -o Joined_Output.mp3 ;
sleep 2 ;
rm Track*.mp3 ;

Filed under: Audio , arecord, ham radio, lame, mp3splt, mpgjoin, pulseaudio, scanner recorder, voice operated recording, vox

Working with Modules in Linux

I use Ubuntu on my desktops because I like the fact that it is a fully built desktop distro with all the goodies already setup for you.  No need to mess with X, etc..  However no distro is perfect, I tend to find many software packages in the repositories are outdated, crippled, missing, buggy, and so on.  A lot of times I’ll also find that hardware support is missing, or Ubuntu’s version is buggy.  Anyway, as long as you know how to compile and install software and drivers, then these are non-issues.

Say you have some hardware which is not supported, or Ubuntu’s driver is buggy.  First you need to find drivers, for example, the Atheros wifi driver in Ubuntu has bugs that cause system freezes, or I want to get my webcam working.  So you need to do this research first.  Then just follow the instructions for compiling and installing the module.

In the case of the webcam module, after compiling the module was installed at

/lib/modules/2.6.27-11-generic/extra/ov51x-jpeg.ko

To load this module I type:

sudo modprobe ov51x_jpeg
then run “dmesg” and you should see the messages that the hardware was found and registered, assuming you had the hardware connected when you loaded the module.

You should also run “lsmod” to make sure your module is loaded.  Something like “lsmod | grep ov” with provide an output showing modules with “ov” in their name.  I do that just to limit the output and make it easier to find specific things.

To unload a module:

sudo modprobe -r MODULE

Most if not all modules have options available to them.  You may be wondering where a lot of options for hardware devices are hiding in Linux.  This is the place.  In order to see what options are available just type:

modinfo MODULE

In the case of my webcam, this showed me about 25 options ranging from control of the LED to the autogain and backlighting controls and so forth.  Very useful to say the least.

As root, edit the file “/etc/modprobe.d/options” and add a line like this:

options MODULE OPT1=x OPT2=x

where MODULE is the module name, and OPT1, OPT2 and so on, are the names of the options you got when running modinfo.  So a real world example would be like “options ov51x_jpeg led=1 autogain=0 backlight=1″

In order to figure out what variables these options will take, sometime it will tell you when you run modinfo, other times you might need to go to the driver documentation or the source code.  For example, in the *.c files (the source code) for the webcam driver, the devs were nice enough to include lots of comments about what the options do and how to use them.

Note that the options that you add to /etc/modprobe.d/options will always apply whether you load the module manually or during boot.  If you need to temporairly change or disable the options, just comment out the line, then unload and reload the module.

In closing, once you are satisfied that the newly compiled and installed module is working and you have the options set to your needs, to automatically load the module during bootup, as root edit “/etc/modules” and add the module’s name to the bottom of the file.

Filed under: Uncategorized , /etc/modprobe.d/options, /etc/modules, compiling, modules, ubuntu

Linux Emulator Frontends

As I mentioned in another post, I really like this new SDLMAME frontend called MAMEPGUI. The author just updated it with some really nice features.

I also just discovered a nice comprehensive frontend called Gelide. This is one of those frontends that you see often in Windows which gathers all your ROMs and emulators into one frontend and lets you launch the ROMs in the appropriate emulator, kind of like a frontend to many other frontends. Similar to QuickPlay for Windows.

It’s a very early release, but it looks like the dev has a great deal of features covered; snaps, customization of emulators, parameter customization, easy to understand interface, etc.. I haven’t played with it much yet, but what I have tried so far has impressed me. For example, I noticed that the default setup doesn’t have anything for C64 emulation. So all I had to do was right click on the games library and add an entry for the Commodore 64, then in the lower box I right clicked and added a definition and parameters for the emulator I use. Then I told it the directory to find the ROMs and supporting files, refreshed the list, then double clicked a game and off it went!! Very nice!!

Filed under: Emulation , Emulation, gelide, linux, mamepgui, sdlmame

Creative Labs Live! IM Webcam (VF0350) in Linux

I just purchased this webcam to use with Skype.

Creative Labs VF0350 USB cam with mic

This cam identifies itself as Device ID 041e:4067 Creative Technology, Ltd.

Even though I run a 2.6.27 kernel which is supposed to have support for many webcams, I still needed to jump through a few hoops to make it work. Firstly, I followed this guide to install a hacked ov51x_jpeg driver. http://www.ubuntuhcl.org/browse/product+creative-labs-live-cam-video-im-vf0350?id=4929

If using only for Skype, DO NOT add “options ov51x_jpeg force_rgb=1″ to /etc/modprobe.d/options as explained in the procedure, the red-blue color issue is not a problem in Skype, only in other applicatons. If the cam is needed outside of Skype, then using “force_rgb=1″ when loading the module will be necessary.

To manually load the module and test the camera use “modprobe ov51x_jpeg forceblock=1″

“lsmod | grep ov” shows:

ov51x_jpeg 159224 0
videodev 41344 5 ov51x_jpeg,msp3400,tuner,ivtv,bttv
usbcore 149360 10 ov51x_jpeg,snd_usb_audio,snd_usb_lib,usbhid,usblp, usb_storage,libusual,uhci_hcd,ehci_hcd

For automated loading of the module during bootup (with the correct options for Skype), edit /etc/modprobe.d/options to include “options ov51x_jpeg forceblock=1″ and edit /etc/modules with “ov51x_jpeg”.

Replace “forceblock=1″ with “force_rgb=1″ in the /etc/modprobe.d/options file if NOT using with Skype.

Filed under: Uncategorized , creative labs live, ov51x_jpeg, vf0350, webcam

Printing to PDF broken in Ubuntu Intrepid

Ubuntu did something weird with the CUPS-PDF thing in Intrepid.  It’s broken when you try to print a file to the PDF output.  It’s related to this bug which has something to do with AppArmor. You’ll get a message “/usr/lib/cups/backend/cups-pdf failed” if you try to print a test page from the CUPS admin server, and you’ll see these messages in /var/log/messages:

type=1503 audit(1227758604.902:61): operation="capable" name="dac_override" pid=16910 profile="/usr/lib/cups/backend/cups-pdf"
type=1503 audit(1227758604.902:62): operation="capable" name="dac_read_search" pid=16910 profile="/usr/lib/cups/backend/cups-pdf"
type=1503 audit(1227758632.546:63): operation="capable" name="dac_override" pid=16953 profile="/usr/lib/cups/backend/cups-pdf"
type=1503 audit(1227758632.546:64): operation="capable" name="dac_read_search" pid=16953 profile="/usr/lib/cups/backend/cups-pdf"
type=1505 audit(1227758866.162:65): operation="profile_load" name="/bin/ping" name2="default" pid=17158
type=1505 audit(1227758866.226:66): operation="profile_replace" name="/usr/share/gdm/guest-session/Xsession" name2="default" pid=17164
type=1505 audit(1227758866.250:67): operation="profile_load" name="/sbin/klogd" name2="default" pid=17168
type=1505 audit(1227758866.294:68): operation="profile_load" name="/sbin/syslog-ng" name2="default" pid=17171

I was fooling around a bit and was able to get it working again by doing the following. Warning, this solution might be considered “hackish”, I don’t fully understand apparmor and apparmor profiles, I just want to be able to print to PDF… Is that asking too much!?! I don’t know what security risks this might pose if you are on a multi-user network or run servers. I don’t worry since I’m the only one on my network and I’m behind a stealthed router and don’t run any servers to the outside.

In a root terminal:

apt-get install apparmor-profiles
ln -s /etc/apparmor.d/usr.sbin.cupsd /etc/apparmor.d/disable/
apparmor_parser -R /etc/apparmor.d/usr.sbin.cupsd
/etc/init.d/cups restart


Filed under: Uncategorized , cups, intrepid, pdf printer

The Power of PulseAudio and JACK

If you use Ubuntu, then they chose for you to use PulseAudio as a layer on top of Alsa by default. It was a good choice.  At first glance the power of PulseAudio is not readily apparent. It’s not really clear what it can do for you other than further confuse the messy state of audio on Linux to begin with.

It’s actually a very powerful layer that allows you to control internal audio streams much like a patchbay or mixer of inputs and outputs and the wires that connect them together. It’s also useful to workaround bugs in Alsa. =) For example, the Intel HDA audio controllers are well known for having bugs with Alsa. Personnally I have experienced these Alsa bugs related to low volume, recording, and problems with capture mixer settings and so forth. PulseAudio allows me to work around these issues by using Pulse to re-route my audio streams to where I want them to be.

I recommend reading this guide and following each item step by step. It has helped me a great deal to understand how Pulse and Jack work.  You really do need to spend the 30 minutes or so to read through that guide and make the changes that are discussed, Pulse and Jack won’t work properly unless you do.

Once you have done that, I have outlined some examples below of using Pulse and Jack along with programs such as gtk-recordMyDesktop which will allow you to make screen captures along with the internal audio mix, as well as making audio-only captures of the “stereo mix” (audio that comes out of your speakers) with programs like arecord, and doing real time MP3 capture of this audio.  The uses for these tips are endless, especially if you need to capture some internet media that can’t be grabbed by sniffing out a URL.

GTK-RecordMyDesktop video grabbing with audio:

Start JACK Control and start the Jack server.

Open the application that will play the audio/video (your web-browser for example) and then using the PulseAudio Volume Control, move the browser’s playback stream to the Jack sink.

Open one each of the PulseAudio playback and recording VU meters, move both of their streams to the PulseAudio JACK Sink.

In output devices, set the Jack sink as default. In input devices, set the Jack source as default.

In gtk-recordmydesktop, set the audio to be captured from JACK, select both system:capture_1 and system:capture_2.

Set up your recordmydesktop window and start the recording, open the Jack connection box and connect the PulseAudio JACK Sink to the recordMyDesktop input port.

Now you should end up with a video that contains the same audio that you heard through your speakers.  You have to use the Jack method with recordmydesktop because apparently it only works with alsa and oss, and gtk-recordmydesktop is only acting as a frontend to write out the command line.  It would be an interesting experiment to try using the Pulse wrapper (padsp) but anyway the Jack method works perfectly and is easier to do with the GUIs.

Stereo Mix audio capture using PulseAudio routing:

Start the playback of the audio source, for example an audio stream coming from some website.

In the PulseAudio Volume Control, set the ALSA PCM as the default in the input and output devices.

Open one each of the PulseAudio playback and recording VU meters, move the stream of the recording meter to the Monitor Source of the ALSA PCM. As the audio is playing, you should see both meters response following each other.

Now you are ready to record.  Use arecord without specifying a device name (-D).

arecord -f cd -vv test.wav

arecord -f cd -t wav | lame -h -V2 – high_quality_output.mp3

arecord -f cd -t wav | lame -m s -a -b 64 – mono_low_44khz_output.mp3

arecord -f S16_LE -c1 -r22050 -t raw | lame -r -s 22.05 -m m -b 64 – mono_low_22khz_output.mp3

The first command will grab the audio with “CD” quality (44.1 kHz, stereo).  The second command is an example of piping arecord’s capture into lame and making a real-time MP3 with the -V2 vbr quality.  The other commands are just further examples of using different sampling rates, mono, low bitrate, etc..  Read the man pages for arecord and lame and you can make up your own commands as desired.

Filed under: Audio , arecord, Audio, jack, mp3, pulseaudio, recording, recordmydesktop, stereo mix

Streamripping… Linux Style

Many times you’ll want to save audio and video streams that you find on websites.  Youtube or Google Video clips are an example, perhaps another example would be some videos on a commercial or media conglomerates website.  There are also commercial radio station websites that offer up streams of past radio shows.  Usually in the blogger and open source arena it’s not a problem to find a link to download the media, because these folks believe in allowing you to use the media as you want under Creative Commons or whatever.

Anyway, here are some tips to find the embedded links in streaming media.  Note, all of these progs need to be run as the root user.

Streamsniff.  This is very nice application, and easy to use.  Just play the stream in your browser and fire up streamsniff in a terminal, no CLI switches necessary.  Hopefully it finds the URL for you.

“tcpick” available in most distro repositories.  Try these commands and see if they give you anything useful.

tcpick -i eth0 -C -a
tcpick -i eth0 -yP | grep GET
tcpick -i eth0 -yP | grep -i flv
tcpick -i eth0 -yP | grep -i mms

You can see what I’m trying to do here, I’m looking for text like “mms” or “flv” to try and find the stream URL.

“tcptrack” available in most repositories.  Run “tcptrack -i eth0″ and maybe it will help you.

“youtube-dl” this is a nice one for downloading YouTube videos from the command line, syntax is “youtube-dl -o filename.flv URL”

If you find an RTSP transport stream you can use the mplayer to save it to a local file:

mplayer -noframedrop -dumpstream “rtsp://www.url.to.stream.file/etc/etc” -dumpfile /home/user/output.rm

Here’s a useful tip for saving audio streams if you have the URL to a Real Media file (you’d be surprised how often you still see corporate entities and “old media” websites still using Real Media!):

mplayer -cache 1024 -playlist http://www.url.to.stream/file.ram -vc null -vo null -ao pcm:fast:waveheader:file=output.wav

I don’t want to leave out stuff like Shoutcast managers, Tunapie is my favorite.  It has a GUI to allow you to rip audio streams with streamripper.

If you are a fan of Last.fm then you must try Vagalume, this version has a save patch.

These tips also wouldn’t be complete unless I mentioned the obligatory Firefox extensions to save videos from YouTube, Google, Blip, etc etc etc…..  You can find these for yourself.

Sometimes you find website streams in which none of these methods work.  Think of sites like big media, hulu, TV networks, etc..  This usually means that there is no URL to a single media file, the stream is actually made up of thousands of small files that get pieced together dynamically, you can see this if you watch the stream in a packet sniffer.  Quite ingenious if I do say so.

Well, in these situations you’ll need to do a screen capture along with the stereo mix audio capture.  I’ll be discussing this in my next post.

Filed under: Audio , embedded urls, ripping, stream, streamsniff, streamtuner, tcpick, tunapie

Ubuntu Intrepid Upgrade, Office 2007 under Wine

I just upgraded one of my boxes from Ubuntu Hardy to Intrepid. It actually worked very well and I didn’t have any problems with the core system afterwards. For some reason a few emulators I had installed with debs were uninstalled while others weren’t… I don’t get it..

The only thing that broke (and perhaps for the better) was my old copy of Crossover Office. I used it for one purpose, to run the now ancient Office 2000 for my wife. Anyway, the apps like Word and Excel would just keep crashing after a few seconds of using them, and there’s nothing on Codeweaver’s website about it, and I couldn’t find anything on Google. When I installed Hardy, I had the same problems, but the fix back then as documented by Codeweaver’s was to edit /etc/sysctl.conf and make the following change:

vm.mmap_min_addr = 65536

to

vm.mmap_min_addr = 0

Apparently this doesn’t work anymore due to this issue. Anyway, my wife had bought a copy of Office 2007, and I found this very nice post explaining how to install it with regular Wine which ended up working like a charm!

Filed under: Wine , codeweaver's, crossover, office, Wine