Dec
04

In the past I posted a few tips and scripts for doing voice operated audio recording (VOX) under Linux.  I’ve done more research on this over time and now have what I believe to be a more refined version as well as finally figured out how to make a timed script with the help of folks from LinuxQuestions.org.

#!/bin/bash
read -p "How many seconds to run? ==> " count_secs
if [[ $count_secs -gt 0 ]]
then
rec -r 22050 -c 1 -p | sox -p OUTPUT.mp3 silence 1 0.1 1% -1 0.5 1% &
my_PID=$!
sleep $count_secs
kill -15 $my_PID
fi

When you run the above script it will ask you to input the total recording time in seconds, then end itself when that time expires. I’ve hard coded the sox command to output a 22 kHz, mono, 32 kbps MP3 file. There are still many many ways to modify and improve this script, but this is working for my needs now.

I’ve also learned a lot more about how to use the “silence” option in sox thanks to this blog post which I suggest you read to understand how to tweak the parameters.  The way I hardcoded the volume thresholds and silence detection time works well to my ear as far as cutting silence from scanner radio traffic. YMMV.

I have one other revised script which first uses arecord and then sends the output to sox for trimming.  The method below has an advantage in that you can more finely tweak the initial capture parameters.  The disadvantage to this method is that there is a second separate processing step with sox, instead of doing it all in one shot as in the first script.

#!/bin/bash
#USAGE: ./sox_vox_recorder.sh xxx output.mp3
#USAGE: Records from line-in for xxx seconds to output.mp3, then detects and trims silence
TIME=$1
OUTPUT=$2
arecord -f S16_LE -c1 -r22050 -t raw -d ${TIME} | lame -r -s 22.05 -m m -b 64 - ${OUTPUT} ;
sleep 1 ;
sox ${OUTPUT} ${OUTPUT}_processed.mp3 silence 1 0.1 1% -1 0.5 1% ;

I can see lots of tweaks to both of these scripts, such as capturing with OGG, or making a case-selection menu to choose capture parameters, and so forth. Maybe I’ll play with this in the future.

Nov
24

Thought I’d post this as I haven’t found all this information in one place.  Sometimes I like to take random movies I’ve downloaded off the net and encode and burn them to standard NTSC DVD.  The problem is a lot of the videos you find are from PAL (25 fps) source and/or they use out of the ordinary aspect ratios like 2.45:1 or 1.91:1.  These two issues can give you major headaches when trying to get them onto DVD and preserve the proper aspect ratio and smooth frame rate.

There’s a number of solutions to these problems which I’ll outline below.

Convert “Any” PAL to NTSC MPEG2-PS (DVD), Soft-Telecine method

mencoder INPUT_PAL.avi -vf scale=720:480,harddup -srate 48000 /
-af lavcresample=48000 -ovc lavc -oac lavc -of mpeg -lavcopts / vcodec=mpeg2video:vrc_buf_size=1835:vrc_maxrate=9800: /
vbitrate=5000:keyint=15:vstrict=0:acodec=ac3:abitrate=224: /
aspect=16/9 -mpegopts format=dvd:tsaf:tele_src=25: /
tele_dest=30000/1001 -o OUTPUT_NTSC.mpg

This “soft-telecine” method flags the frames for 25 fps to 29.97 fps to make the file NTSC compliant without changing the actual frame rate.

Dealing with the Aspect Ratio problem

The mencoder command above is incomplete because if you’ll notice it assumes you have a source already encoded at 720×480 with a pixel aspect of 1:1 and a display aspect of 16:9.  Of course this is never the case, so you need to make a few modifications.  We need to do some simple math.

Example:  Source file has a display aspect of 2.45:1 and a resolution of 640×261.

Desired output is NTSC-DVD with 16:9 display aspect:  (480)*(16/9)=854, and (854)/(2.45)=349, so we’ll encode to 720×350 and pad/expand the remainder to 480.

mencoder uses the expand method, portions of the above command line would therefore be modified like this “-vf scale=720:350,expand=720:480″ and make sure the -lavcopts part has “aspect=16/9″ in it somewhere.

If you happen to prefer ffmpeg, it uses the padding method and you would achieve the same thing by having the following in the command line “-s 720:350 -padtop 65 -padbottom 65 -aspect 16:9″.

Nov
21

I just updated my main box to Karmic this past week.  This is a 32 bit box that was first installed with Hardy.

Overall it looks like they improved a bunch of stuff.  I noticed a lot of stupid annoyances with Gnome/Nautilus are gone now, and the new Pulseaudio stuff looks good.

I use this box as my main PC as well as MythTV frontend/backend with one other PC on the network as a frontend only.  My capture card is the Hauppauge PVR-250.  I run a heavily patched version of 0.21-fixes from SVN built to my needs.  For Karmic I had to update it to r20877 due to some weirdness with glibc 2.10+, anyway that was the easy part…

After compiling and attempting to run it, I wasn’t getting any audio, and the video was choppy.  My first thought was to blame pulseaudio due to the debug messages I was getting and also the fact that if I changed my “profile” in the new gnome-volume-control panel it would make audio come and go, but this fact actually turned out to be a semi red herring.

After a little investigating I discovered to my surprise that grabbing video directly from the PVR-250 via “cat /dev/video0 > output.mpg” was also producing the same screwed up video/audio as Myth live TV.  Then I checked the behavior of my frontend only machine, same problem there…  Strange, what could be causing this?  IVTV it must be.  Jaunty had IVTV module v1.4.0, Karmic has v1.4.1 which appears to be non-stable at the present time.

So I wanted to try and compile a 1.4.0 module for Karmic but couldn’t find the source on the IVTV page, so I gave up on that.  Anyway long story short, I ended up solving these very strange problems by doing the following:

1)  I found out that for some very strange reason, the ivtv module needs to be re-loaded AFTER the mythbackend is started!  WTF!  and why does this fix the problem?  Any feedback on this would be appreciated.  Does it have something to do with this weird new Karmic thing where they start all the init programs in strange order to improve the boot speed?

2)  Declare EXPERIMENTALLY_ALLOW_PULSE_AUDIO=1 in my .profile.

All is well again.  Now to go enjoy Karmic.

UPDATE: I finally figured out what was giving me the problem with ivtv, it’s PEBKAC. Back in Feisty (I think) I added the command “v4l2-ctl -b off” to my mythbackend init script. I did this as a workaround to an Xv rendering playback bug with VLC and recordings made from PVR cards with the VBI closed-captioning enabled. Anyway, I guess with the newer ivtv module it screws things up, because now I removed that command and all is back to normal.

Mar
02

This might be of some use to folks running Gnome 2.24 in Ubuntu or Fedora or whatever.  I was recently trying to debug a very annoying problem where every minute or so my MythTV playback would get jittery with a corresponding CPU spike in xorg usage.  The same thing would happen in other video players like VLC.  It didn’t make any difference what kind of output rendering was being used.

It turns out this extremely annoying problem is due to stupid gnome-screensaver!  At least it was on my machine.  Take a look at this.  It’s supposed to be fixed in Gnome 2.26+.  Well, I just thought I’d put this one out there, because it’s very difficult to debug, and I see occasional posts about jittery video playback in Linux.  Of course, a great majority of those problems are due to proprietary video drivers though.

Jan
20

After I made my last post about how to do line-in recordings from your scanner radio with silence detection and cutting, also known as VOX recording in the old days, I started looking at SoX the Linux Sound eXchange CLI audio processing application. Using SoX is a lot slicker than my arecord-mp3splt-mpgjoin method. It lets you do the recording, silence detection, and mp3 encoding in one line, and it puts everything into one audio file, no muss no fuss.

Here are some examples:

rec -r 44100 -p | sox -p OUTPUT.mp3 silence -l 1 00:00:00.5 -45d -1 00:00:00.5 -45d

rec -r 22050 -c 1 -p | sox -p OUTPUT.mp3 silence -l 1 00:00:00.5 -45d -1 00:00:00.5 -45d

The first one will give you a 44.1 kHz, stereo, 128 kbps MP3. The second one will give you a 22.05 kHz, mono, 32 kbps MP3.

The rec command does the recording, -r specifies the sampling rate and -c sets the amount of audio channels, “-p” means the input is piped to another command. The sox pipe also needs a “-p” to tell it that its input is coming from a pipe, in this case the rec command.

The crypt CLI switches in the piped sox command “silence -l 1 00:00:00.5 -45d -1 00:00:00.5 -45d” is what does the silence detection and trimming. It means to cut silence from the beginning of the file and parts within the middle and end. “xx:xx:xx.x” is in hh:mm:ss.s format and tells sox the minimum duration to consider valid audio. In my examples I have this set to 0.5 seconds for valid audio. The “-45d” means -45dB and is the minimum valid audio level to do trimming. In other words, if you were to say -100d, basically everything would be considered valid and there would be no trimming, you will need to play with this depending on your input levels.

I made a test for almost 2 hours with my scanner connected and scanning the aircraft bands. I ended up with a 22 minute file. Very nice!! Now I’d say we have a very nice Linux-style “scanner recorder” program, and it’s a hell of a lot slicker than anything you’d find in Windows.

If you are a Debian or Ubuntu user, you will need to compile your own version of SoX if you want to encode to MP3. This bug explains that the version of SoX in the Debian repos was compiled without MP3 support. Anyway, all you need to do is download the latest source, make sure you have all the build deps (apt-get build-dep sox), make sure you have the libmp3lame and libmp3lame-dev packages installed, then just do ./configure, make, make install.

When you do the ./configure you’ll see a list at the end of the output of what capabilities will be compiled into sox, just make sure that it says you will have MP3 support. Your compiled version will go into /usr/local by default, so you don’t need to uninstall the repo version of sox, since that one is in /usr and programs in /usr/local always take precedance. You can verify that by doing “which sox” after installing it and it should show you “/usr/local/bin/sox”.

Jan
14

Many ham radio operators and scanner enthsiasts have a need to do long time recordings from a line-in source (such as a scanner or ham radio), but they don’t want to record all of the silence in between the transmissions.  Enter the concept of VOX recording.  Those of you that used to use tape recorders back in the good ol days will remember the VOX detection circuit that the higher end recorders used to have.  So, how do we do this with a computer?  In Windows there are a number of applications, Scanner Recorder and Rec-All Pro come to mind.  These apps let you record from the line in and pause the recording when the levels go below a user specified threshold.  They also let you record directly to MP3 as well as set the sampling and bit rate, for compression of the recording.

In Linux I’ve been able to get Rec-All Pro to work, but it’s kind of a kludge since you have to use Wine.  It works, but you cannot encode directly to MP3, I assume this is because of the way the Wine programs use DLLs.  I never was able to figure it out.  Anyway, I prefer to use a Linux native method.

I start off by setting up the radio to the station I want to record, then I use the PulseAudio VU meter along with the Alsa Mixer to set my Line-In level and Capture level to the proper amplitude, by watching feedback from the VU meter.  Then I execute the following to begin the recording:

arecord -D plughw:0,0 -f cd | lame -m s -a -b 64 - FILE.mp3
or
arecord -D plughw:0,0 -f S16_LE -c1 -r22050 -t raw | lame -r -s 22.05 -m m -b 64 - FILE.mp3

The first command will produce a file called “FILE.mp3″ with 16 bit resolution, 44.1 kHz sampling, mono. The second command will encode to 16 bit, 22 kHz, mono.

When you are finished recording do a Ctrl-C to end arecord. Now we need to do the silence detection.

mp3splt -s -p th=-30,min=1,rm FILE.mp3

The command above will split the input file using a silence threshold of -30 dB, a minimum silence duration of 1 second, and removes the silence from the resulting split files. And you will end up with a directory full of “Track 0*.mp3″ files. Now we just need to join all these files into one file. So go ahead and execute:

mpgjoin *.mp3 -o JOINED.mp3

Below is a little script to automate all of this. The only thing you need to change before running it is the bit in the arecord command “-d 300″. Change the 300 part to however many seconds you want to run the recording for.

#!/bin/sh
#USAGE: Records from line-in for -d xx seconds, then splits tracks removing silence between tracks, and combines all into one file
arecord -D plughw:0,0 -f S16_LE -c1 -r22050 -t raw -d 300 | lame -r -s 22.05 -m m -b 64 - Recording.mp3 ;
sleep 3 ;
mp3splt -s -p th=-30,min=1,rm Recording.mp3 ;
sleep 2 ;
mv Recording.mp3 Recording.mp3.original ;
sleep 2 ;
mpgjoin *.mp3 -o Joined_Output.mp3 ;
sleep 2 ;
rm Track*.mp3 ;

Jan
08

I use Ubuntu on my desktops because I like the fact that it is a fully built desktop distro with all the goodies already setup for you.  No need to mess with X, etc..  However no distro is perfect, I tend to find many software packages in the repositories are outdated, crippled, missing, buggy, and so on.  A lot of times I’ll also find that hardware support is missing, or Ubuntu’s version is buggy.  Anyway, as long as you know how to compile and install software and drivers, then these are non-issues.

Say you have some hardware which is not supported, or Ubuntu’s driver is buggy.  First you need to find drivers, for example, the Atheros wifi driver in Ubuntu has bugs that cause system freezes, or I want to get my webcam working.  So you need to do this research first.  Then just follow the instructions for compiling and installing the module.

In the case of the webcam module, after compiling the module was installed at

/lib/modules/2.6.27-11-generic/extra/ov51x-jpeg.ko

To load this module I type:

sudo modprobe ov51x_jpeg
then run “dmesg” and you should see the messages that the hardware was found and registered, assuming you had the hardware connected when you loaded the module.

You should also run “lsmod” to make sure your module is loaded.  Something like “lsmod | grep ov” with provide an output showing modules with “ov” in their name.  I do that just to limit the output and make it easier to find specific things.

To unload a module:

sudo modprobe -r MODULE

Most if not all modules have options available to them.  You may be wondering where a lot of options for hardware devices are hiding in Linux.  This is the place.  In order to see what options are available just type:

modinfo MODULE

In the case of my webcam, this showed me about 25 options ranging from control of the LED to the autogain and backlighting controls and so forth.  Very useful to say the least.

As root, edit the file “/etc/modprobe.d/options” and add a line like this:

options MODULE OPT1=x OPT2=x

where MODULE is the module name, and OPT1, OPT2 and so on, are the names of the options you got when running modinfo.  So a real world example would be like “options ov51x_jpeg led=1 autogain=0 backlight=1″

In order to figure out what variables these options will take, sometime it will tell you when you run modinfo, other times you might need to go to the driver documentation or the source code.  For example, in the *.c files (the source code) for the webcam driver, the devs were nice enough to include lots of comments about what the options do and how to use them.

Note that the options that you add to /etc/modprobe.d/options will always apply whether you load the module manually or during boot.  If you need to temporairly change or disable the options, just comment out the line, then unload and reload the module.

In closing, once you are satisfied that the newly compiled and installed module is working and you have the options set to your needs, to automatically load the module during bootup, as root edit “/etc/modules” and add the module’s name to the bottom of the file.

Jan
04

As I mentioned in another post, I really like this new SDLMAME frontend called MAMEPGUI. The author just updated it with some really nice features.

I also just discovered a nice comprehensive frontend called Gelide. This is one of those frontends that you see often in Windows which gathers all your ROMs and emulators into one frontend and lets you launch the ROMs in the appropriate emulator, kind of like a frontend to many other frontends. Similar to QuickPlay for Windows.

It’s a very early release, but it looks like the dev has a great deal of features covered; snaps, customization of emulators, parameter customization, easy to understand interface, etc.. I haven’t played with it much yet, but what I have tried so far has impressed me. For example, I noticed that the default setup doesn’t have anything for C64 emulation. So all I had to do was right click on the games library and add an entry for the Commodore 64, then in the lower box I right clicked and added a definition and parameters for the emulator I use. Then I told it the directory to find the ROMs and supporting files, refreshed the list, then double clicked a game and off it went!! Very nice!!

Jan
04

UPDATE 1/19/10: The latest Ubuntu Karmic kernel 2.6.31-18 has support for this camera via gspca. After updating to the latest kernel, just reboot and the module should load automatically. If you do a “lsmod |grep gspca” it should show 2 modules loaded, gspca_main and gspca_ov519.

You’ll probably notice that when you try to actually use the webcam, it just gives you a green or black screen output. This is becuase it needs the v4l1 compatibility library to be loaded before use. Specify the environment variable LD_PRELOAD=/usr/lib/libv4l/v4l1compat.so before using the webcam. On a command line it would be like this:

env LD_PRELOAD=/usr/lib/libv4l/v4l1compat.so skype

env LD_PRELOAD=/usr/lib/libv4l/v4l1compat.so camstream

The lines above are an example of how I would run Skype or camstream, for example. You can use the same line in a Gnome or KDE launcher menu item as well. The v4l1compat.so library comes from a certain package on Ubuntu so you may not have it installed, but I’m not at my box right now to figure out which package. You can do an “apt-file search v4l1compat.so” to find out which package it comes from.

UPDATE 12/5/09:  The instructions below no longer work in kernels 2.6.31+ at least with Ubuntu Karmic.  This hacked driver is no longer maintained and won’t compile on recent kernels, but support for this camera was added in the lastest v4l-dvb gspca_ov519 driver.  Refer to https://bugs.launchpad.net/ubuntu/+source/linux/+bug/492744 and the patch at http://linuxtv.org/hg/v4l-dvb/rev/a130fb09f7bafe1ea0e9ace9d813a92327d704ec

You can download and install a fresh pull of v4l-dvb and compile/install your own, then modprobe gspca_ov519 and it should work.  Or hopefully the Ubuntu devs will add this patch to the next kernel release.

I just purchased this webcam to use with Skype.

Creative Labs VF0350 USB cam with mic

This cam identifies itself as Device ID 041e:4067 Creative Technology, Ltd.

Even though I run a 2.6.27 kernel which is supposed to have support for many webcams, I still needed to jump through a few hoops to make it work. Firstly, I followed this guide to install a hacked ov51x_jpeg driver. http://www.ubuntuhcl.org/browse/product+creative-labs-live-cam-video-im-vf0350?id=4929

If using only for Skype, DO NOT add “options ov51x_jpeg force_rgb=1″ to /etc/modprobe.d/options as explained in the procedure, the red-blue color issue is not a problem in Skype, only in other applicatons. If the cam is needed outside of Skype, then using “force_rgb=1″ when loading the module will be necessary.

To manually load the module and test the camera use “modprobe ov51x_jpeg forceblock=1″

“lsmod | grep ov” shows:

ov51x_jpeg 159224 0
videodev 41344 5 ov51x_jpeg,msp3400,tuner,ivtv,bttv
usbcore 149360 10 ov51x_jpeg,snd_usb_audio,snd_usb_lib,usbhid,usblp, usb_storage,libusual,uhci_hcd,ehci_hcd

For automated loading of the module during bootup (with the correct options for Skype), edit /etc/modprobe.d/options to include “options ov51x_jpeg forceblock=1″ and edit /etc/modules with “ov51x_jpeg”.

Replace “forceblock=1″ with “force_rgb=1″ in the /etc/modprobe.d/options file if NOT using with Skype.

Nov
27

Ubuntu did something weird with the CUPS-PDF thing in Intrepid.  It’s broken when you try to print a file to the PDF output.  It’s related to this bug which has something to do with AppArmor. You’ll get a message “/usr/lib/cups/backend/cups-pdf failed” if you try to print a test page from the CUPS admin server, and you’ll see these messages in /var/log/messages:

type=1503 audit(1227758604.902:61): operation="capable" name="dac_override" pid=16910 profile="/usr/lib/cups/backend/cups-pdf"
type=1503 audit(1227758604.902:62): operation="capable" name="dac_read_search" pid=16910 profile="/usr/lib/cups/backend/cups-pdf"
type=1503 audit(1227758632.546:63): operation="capable" name="dac_override" pid=16953 profile="/usr/lib/cups/backend/cups-pdf"
type=1503 audit(1227758632.546:64): operation="capable" name="dac_read_search" pid=16953 profile="/usr/lib/cups/backend/cups-pdf"
type=1505 audit(1227758866.162:65): operation="profile_load" name="/bin/ping" name2="default" pid=17158
type=1505 audit(1227758866.226:66): operation="profile_replace" name="/usr/share/gdm/guest-session/Xsession" name2="default" pid=17164
type=1505 audit(1227758866.250:67): operation="profile_load" name="/sbin/klogd" name2="default" pid=17168
type=1505 audit(1227758866.294:68): operation="profile_load" name="/sbin/syslog-ng" name2="default" pid=17171

I was fooling around a bit and was able to get it working again by doing the following. Warning, this solution might be considered “hackish”, I don’t fully understand apparmor and apparmor profiles, I just want to be able to print to PDF… Is that asking too much!?! I don’t know what security risks this might pose if you are on a multi-user network or run servers. I don’t worry since I’m the only one on my network and I’m behind a stealthed router and don’t run any servers to the outside.

In a root terminal:

apt-get install apparmor-profiles
ln -s /etc/apparmor.d/usr.sbin.cupsd /etc/apparmor.d/disable/
apparmor_parser -R /etc/apparmor.d/usr.sbin.cupsd
/etc/init.d/cups restart