Saturday, December 26, 2009

How To: MusicMagicMixer

Another duplicate mp3 finder is MusicMagicMixer. I liked this one for speed and simple UI. On launching the application you will be provided with a dialog to select the media you like to scan.



After selecting the directory containing the media it will start scanning the files.



And once that is done the application will start processing the files to create audio fingerprints immediately.



The good thing is that the processing is very fast. In about 20 minutes it was able to finish processing all of the 578 files. However the bad news is that it is not as good as DuMP3 when finding duplicates. It found none from the 578 files.




So you either have a fast but not accurate software or accurate but slow software. The choice is yours :). But for me I still like the MusicMagicMixer. It did find duplicates in the previous run which I deleted and so now it is showing empty duplicates. And then  DuMP3 could find a few more. I don't know if there is a better software that will find all the duplicates much faster. The search continues...

How To: DuMP3

After searching for some mp3 duplicate finders for Linux I finally found two good applications. One of them is DuMP3. In this blog post I will walk through the steps showing screenshots and measuring the performance.

Upon launching the application you will see the following window on your screen.



A very simple and clean interface. On the left side you will have to select the files that you want to compare and add them into the right pane. In my case all my collection is in one place which I added as shown below



Once you press the "Next" button the software will scan all the media in the directories that you selected in your right pane and it will make a list of file extensions that are contained in the directories. These will be shown in the following screen where you can select the file formats that you want to compare.



Seems like the software can find duplicates of images as well. That is neat! Anyway I just want audio de-duplication so I selected mp3, ogg and flac



Pressing "Next" will take you to a screen where you can tweak the comparison algorithm. I left the default values because I did not know how to tweak them yet.



When you are done changing the parameters, just press Next and the software should start scanning your collection. The software seems to be able to use multiple threads to make fingerprints for some audio files but not all. Or so it seems. For the first 130 files or so I saw that the software was using 50% of all 8 CPUs (4 real and 4 virtual due to Intel's HT) of my core i7 processor. But then after a while a strange thing happened. The scanning switched from multi-threaded to single threaded for some reason! That was very unfortunate. This is what is happening with DuMP3 everytime I use. Why would it not use all the processors when it is capable of it? May be only some formats lend to multi-threaded processing? What formats are these? Anyway this is so slow that it is totally unusable. I just ran in on a sample of about 500 files and it took 24 hours so far, yet it could only scan 475 files.



However it seems like it has done a good job at finding duplicate files. Still I cannot use it. I wish I knew what was wrong.

Thursday, December 24, 2009

Are there some good duplicate mp3 finders?

I have a very large music collection. Some of them are duplicates and remixes. I wanted to clean up my collection of duplicates. Being a Linux user and also cheap I wanted to search for a free software for Linux that would scan through my collection and identify duplicates. Only after searching the net for such a software did I find out how difficult it is to get something like that for Linux. A lot of the software is not free, but more importantly they are not available for Linux.

I finally found a few that were either written in java or python and so are platform independent in a way. Of those I found only two applications worthwhile and easy to use (has an easy to understand GUI interface). One of them is called DuMP3 and another is called MusicMagicMixer, both of which are written in java or have parts of java code. I am intentionally not providing links to these applications because they will become broken links pretty quickly. You can do a search for them on your favorite search engine.

Of these two applications I found MusicMagicMixer to be better. Both in terms of interface and performance. DuMP3 is very slow. I did not compare how well they find duplicates because I could not get DuMP3 to ever complete scanning. It starts out slow and then becomes even slower as it processes more and more files. Not sure why. May be I should search for the latest version and try that. Here is a screenshot of DuMP3.



MusicMagicMixer performed much better. I did not time it but I think it was able to create one fingerprint per minute. So each music file took about a minute to be scanned. The nice thing about MusicMagicMixer is that you can run a webserver and using your browser you can look at the duplicate songs. Here is a screenshot of the app running.



I will write a little bit more about how to use each of the two applications in my next couple of blogs. But for now I am happy that I was able to find some software that can find duplicates for my music collection and that works on Linux and is free!

Wednesday, December 09, 2009

Boxee box?

How about that? Just as I was looking for some HTPCs there are more and more options available. The Boxee box looks very nice and very different from any HTPC cases I have seen so far.




More details over at http://www.boxee.tv/box. I wonder why Boxee went with d-link. What ever the reasons it does sport RF remote control, S/PDIF, RCA audio outputs, HDMI, SD, USB, WiFi, and Ethernet. All this fits into a really small package. Just look at the size along side of a coke can.



It will be interesting to see how thing will go with Boxee. Hulu created so many problems to Boxee and it jumped over all the hoops. It is time for some payback. The interface also looks neat. The Boxee media center is a fork of xbmc so we know it is powerful, well done and will play a lot of media formats.



Waiting for its release...

Monday, December 07, 2009

ION platform with PCI slot

As explained in my previous blog, I needed an ION platform with a PCI slot. The slot will hold my already existing TV tuner card. I searched for a nvidia ION motherboard with a PCI slot. I could only find one in newegg. At $160 it is pretty cheap too!

Now I need a case to enclose it. I found this case. It is not the best, but will do for now. The reason for going with this case is because I wanted it to be compact and at the same time should be able to house 2 3.5" hard drives and a 5.25" blu-ray drive. The case is small at 11.80" x 8.70" x 5.1". The cost of this case is $40.

Next up, I need a couple of memory sticks. I am not planning to remove the memory sticks from my desktop PC and put them here because I want to sell the old desktop in a proper working condition. It will have everything except for the TV tuner card. I think 1 GB should be sufficient for the small HTPC especially since it will run linux. A quick search produced results with 1 GB memory sticks costing about $25.

Finally I need a 1TB hard drive. A decent western digital hard drive can be had for about $85. So that completes the HTPC. I am not planning on purchasing additional hard drive or blu-ray/DVD drive at this point simply because I don't need them yet.

The total cost of the HTPC comes to about $310. Not bad at all. With shipping and taxes it still will not be more than $350. I am not sure when I will buy it, but it is on my list of things to buy...

Monday, November 30, 2009

nVidia ION for HTPC

This is an exciting time for buying HTPC devices. They come in really small sizes, they are efficient and yet can playback 1080p video and boot in under 30 seconds. The most exciting hardware (for me anyway) is the ION platform from nVidia. These are tiny motherboard and CPU combos that do wonders considering their size and power usage. I had my eye on this one for a while now. It looks great and for once I can put my HTPC in front of everyone in the living room. But there are a couple of reasons I keep reconsidering. First the bare bones HTPC has a small 2.5" 320 GB hard drive. Well I guess that is what gives it the small size. And the second reason is that it does not have a PCI slot. The tv tuner card I have sits in a PCI slot. Of course I can buy the latest and greatest USB tv tuner card, but I have my reasons to not to.

I have painstakingly configured my PCI tv tuner card in Gentoo (my favorite OS). When I brought the tuner card the drivers were still buggy and you had to use the latest kernel and even then there would be problems. Luckily I use Gentoo so using the latest kernel is no big deal. And since I compile the kernel from scratch I can modify the kernel source code too. And I did just that. I had to tune a few parameters in the v4l driver for my tuner card. Then I had to hack around with mplayer to get everything right. Anyway that was back in the day. The linux drivers have improved a lot since then. At least for the old tuner card that I have, I didn't need to modify the latest kernel anymore. Things just work now.

Yet I don't want to part with the PCI card and go for a USB tuner, because I think the latest tuners will still not be supported by linux yet and I will have to invest a lot of time to get it to work again. And I don't want that. So my next best option would be to use an ION platform that has a PCI slot. That means I will have to build the HTPC from scratch which is fun for me except that it might not look as nice on the exterior. I will probably have to hide it again. At least I will save some on my electric bill if nothing. Running a full desktop PC 24/7 with 4 hard drives and a lot of power hungry components like the very powerful graphics card is not helping my bills :). So in my next post I will try to find an alternative to the barebones ION HTPC and perhaps I will have better luck.

Thursday, November 26, 2009

My HTPC

I have my eye on a new HTPC, but what does my current one look like? Here it is.



The desktop PC that acts as my apache, mysql, samba, vnc, nx, xbmc and mythtv servers. In addition it is a media player. I have to hide it behind my TV because it looks so big and ugly. But it serves me well. Even while running 24/7 for the past two and a half years, I never had a single problem. It has 2 fans, one to cool the processor and one at the back to blow the air out of the case. The fan at the back sometimes would not run after a restart and I have to kick start it by spinning the fan with my hand and it will continue to run. For this reason and also to keep the PC much cooler I have left one side of the desktop open.

This is how my media center running xbmc looks like from the front.



And here is the vnc viewer connected to my HTPC showing off mythtv tv guide.



You can see the gkrellm running on the left side showing all the vital stats like cpu, gpu, hard drive temperature, memory, cpu, hard disk, network usage etc. All in one place. Next up my future HTPC wishlist.

Tuesday, November 24, 2009

The new HTPCs are so tempting

It has been a while since I blogged here. Got very busy. But I am hoping to restart. We will see how it goes.

Today's topic is HTPC. There are some really wonderful HTPCs coming out these days. I wish I could buy all of them :). I have an old HTPC now. You can barely call it a HTPC. It is simply a desktop that I filled up with a lot of hard disks. It has an AM2 motherboard capable of 6 channel surround sound with AMD X2 3600+ processor and nvidia 9800 GT GPU. The GPU is way too powerful than required for a HTPC I know, but it was also my gaming machine for a while. Crammed into all the spaces are 4 hard disks with 250 GB each. I also have a kworld pci ATSC/QAM tuner for all my digital tv recording needs. It was so much fun when I built it more than a couple of years ago.

For the software I used freevo as my media center software. It would record shows too. So I did not need anything else. It was very configurable and everything is in python and most importantly it can playback media using various media players. Of course mplayer handled every kind of media for me. The advantage with this kind of setup is that I need not wait for freevo to pick up the new mplayer code. As soon as I install a new version of mplayer, freevo would start using it. However freevo looked so '90s when it came to graphics and UI design. So I switched to xbmc.

Now xbmc is my favorite media player. It is beautiful and all the animations and skins are a treat to the eyes. But it does not record live tv. So for that I installed mythtv and setup the backend to record live tv. A crobjob faithfully downloads EPG every night. I configure everything in mythtv using the web interface. Easy! The best thing about xbmc is that it uses nvidia's vdpau for decoding and rendering my videos. I love this feature. Of course freevo could also do it too because mplayer can render using vdpau. Anyway the point is that now all the work is offloaded to the GPU and so the CPU is not as taxed.

This and the fact that hard drives with lots of space are dirt cheap brings me to the idea of using a much much lighter setup. What if there is a small low power device that has a nvidia GPU and a low power CPU with just one 1 TB hard drive, has 6 channel analog and a digital audio output and a PCI slot for my tv tuber card? That would be awesome and then I would not have to hide my dirty HTPC behind my LCD TV.

I have my eyes on such a setup. More tomorrow though.

Wednesday, May 06, 2009

Google Chromium (the open source chrome browser) on Gentoo!

Chromium is not yet complete no matter what I say here. Please do not post bugs or assume this is the final version of chromium. It still lacks a lot of features. My intension here is to only show you how you can get started on building chrome in Gentoo.

Google chromium project has come a long way. I have been building chromium once in a while but it was not stable and was slow until recently. But now I am using it as my primary browser (well there are some exceptions as I will explain later)! It is better than my firefox experience (at least in linux).

To get started head over to this link. First you will need to have all the prerequisites. In ubuntu it is pretty easy. On gentoo we need the gentoo package equivalents of all the prerequisites. Here is the list of things I mapped from ubuntu packages to Gentoo.

* Python >= 2.4 ---> python
* Perl >= 5.x --> perl
* gcc/g++ >= 4.2 --> gcc
* g++-multilib >=4.2 --> ?
* bison >= 2.3 --> bison
* flex >= 2.5.34 --> flex
* gperf >= 3.0.3 --> gperf
* pkg-config >= 0.20 --> pkgconfig
* libnss3-dev >= 3.12 --> nss
* libgconf2-dev --> gconf
* libglib2.0-dev --> glib
* libgtk2.0-dev --> gtk-engines-murrine
* libnspr4-0d >= 4.7.1+1.9-0ubuntu0.8.04.5 --> nspr
* libnspr4-dev >= 4.7.1+1.9-0ubuntu0.8.04.5 --> nspr
* msttcorefonts (Microsoft fonts) --> corefonts
* freetype-dev --> freetype
* libcairo2-dev --> cairo
* libdbus-1-dev --> dbus

The packages I left with a question mark are the ones that I did not how they get mapped on gentoo. Make sure you have installed all the packages with the minimum version required as specified. If you are running 64 bit OS you will probably need to install more packages. See here for 64 bit OS. I was able to build chrome in 64 bit ubuntu following those instructions, so I assume it should be fairly easy to build on 64 bit gentoo. Once you have all the dependencies, it is pretty straight forward.

Get the source code from here. I did not bootstrap from the tarball, because I found the svn checkout easier. But I would suggest that you do use the tarball. It will save a lot of time. Believe me. Following svn instructions, you will need to download depot tools. I downloaded the tools to ~/workspace/c/chrome/chromium/depot_tools. So all the instructions will use that directory.

Next you will need to configure the client and download the code using the following commands.

~/workspace/c/chrome/chromium $ ./depot_tools/gclient config http://src.chromium.org/svn/trunk/src http://chromium-status.appspot.com/lkgr

~/workspace/c/chrome/chromium $ ./depot_tools/gclient sync

This will take a really REALLY REALLY long time. Once you have the code you are ready to build.

~/workspace/c/chrome/chromium $ cd src/build

~/workspace/c/chrome/chromium/src/build $ nice -n 10 ../../depot_tools/hammer --mode=Release app

I niced it a little bit to keep my computer usable for other tasks while it is compiling. The compile will go on for a long time. If everything went without a hitch you will have your chrome browser in ~/workspace/c/chrome/chromium/src/sconsbuild/Release/chrome. Yay! Go ahead take it for a ride. You will be pleasantly surprised to see how much already works.

I ran some chrome experiments to gauge the performance and it is pretty good. Perhaps not as good as on windows, but definitely good. I did not do any firefox to chrome comparisons, so cannot comment much on that. The only downside of chrome browser is that it lacks flash at this point. So no youtube for you. But hey I can use firefox for that. A lot of things still don't work. For example ctrl-alt-t doesn't work. A lot of the options don't still work. Bookmarks have problems, etc. etc.

Here are some screen shots of chromium running in linux.


Monday, April 20, 2009

Installing Boxee on Gentoo: my experience

You have probably heard of Boxee and Gentoo (come on). I wanted to give Boxee a try, because of all the movies and tv shows that I can watch for free. I have already switched from Freevo to xbmc (which is a story in itself) as my primary media center. Since boxee is a fork of xbmc I thought I might give it a try. Why not? Because xbmc was so easy to compile and install. I did not have any problem compiling the latest code from subversion. I registered and downloaded the latest code from Boxee's website. There is no link to svn and so I downloaded version 0.9.11.5591.

The compile was pretty simple
1. Untar the downloaded file
2. autoconf
3. ./configure
4. make -j3

However the compile was not successful. The first problem I ran into was this compile error:
XAPP_MC.cpp:12:20: error: bxcurl.h: No such file or directory
To solve it, I googled around and found the solution in the boxee forums. I had to make minor modifications to the following Makefiles
xbmc/Makefile
SRCS += BoxeeAuthenticator.cpp

xbmc/lib/libPython/Makefile
SRCS += XBPyPersistentThread.cpp

xbmc/app/Makefile
SRCS+= XAPP_Edit.cpp
With the updated makefiles I started the compile process again. Now I got this error message:
*** [Gentoo] sanity check failed! ***
*** libtool.m4 and ltmain.sh have a version mismatch! ***
*** (libtool.m4 = 1.5.26, ltmain.sh = 1.5.2) ***

Please run:

libtoolize --copy --force

if appropriate, please contact the maintainer of this
package (or your distribution) for help.

make[2]: *** [config.status] Error 1
make[2]: Leaving directory `/home/chandanp/workspace/c/boxee/boxee-0.9.11.5591-src/xbmc/cores/dvdplayer/Codecs/libmad'
make[1]: *** [libmad] Error 2
make[1]: Leaving directory `/home/chandanp/workspace/c/boxee/boxee-0.9.11.5591-src/xbmc/cores/dvdplayer/Codecs'
make: *** [dvdpcodecs] Error 2
Well the solution is in the error message. I issued the following commands:
cd xbmc/cores/dvdplayer/Codecs/libmad/
libtoolize --copy --force
cd -
make
The project compiled successfully. Now to test it. I ran ./Boxee and everything worked beautifully out of the box. I added my boxee account and logged in. Then I could browse all the online content. Sweet! But there is just one problem. I cannot play any of the content I wanted. When I select a video in comedy central the busy circle shows up and it rotates for ever without starting the video. A look at the output showed these errors/warnings:
./bxflplayer-linux: /usr/lib/libcurl.so.4: no version information available (required by ./bxflplayer-linux)
./bxflplayer-linux: /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by ./bxflplayer-linux)
Once again boxee forums to the rescue. The solution that was offered was to copy bxflplayer-linux from an older 0.9.8.4880 release into the latest. So I downloaded 0.9.8.4880 and copied over all the files from boxee-0.9.8.4880/system/players/flashplayer/ to boxee-0.9.11.5591-src/system/players/flashplayer/.

I restarted boxee and couldn't believe my eyes when I saw the videos playing nicely. Watched some clips from comedy central and then turned my attention to the controversial hulu. I clicked on hulu, and everything seemed normal. So I selected a video clip, the player started but I did not hear any audio and the video was just a blank white screen. But the time was ticking as if the media is playing. I did not have my hopes too high anyway and did not expect hulu to work on boxee, what with all the drama. May be there is something subtle that I am missing or may be the old bxflplayer-linux cannot handle hulu's content. No matter what I will keep looking around for some hacks so I can enjoy hulu from boxee.

That is my experience so far with boxee. Oh by the way I had to emerge a lot of packages along the way to make the ./configure work (you can find some dependencies here but it may not be up to date) and also when the bxflplayer could not find some libs (I had to emerge dev-perl/GSSAPI for example). I hate closed source binaries, they are compiled against some libraries and you don't know which libraries are required. Well ldd to the rescue. Still ./configure is the right way.

Update: If you are seeing floating point exception when playing your local media in Boxee then you will need to re-emerge media-libs/alsa-lib" *with* the debug use flag. Here is the discussion from comments regarding this issue:

Blogger VirtualEnder said...
Interesting, well I linked the built, libmad.0.2.1 to libmad.so, then it built just fine. However I'm still having the issue with it crashing any time I try to play anything. I set the log level to 0 and it doesn't even give any errors. It prints to stdout "floating point exception" and that's it. So bizarre. All the internal stuff works fine, as does the music playback, just seems to fuss with videos. mplayer plays the videos fine by itself, no FPE there, so I'm not sure what the problem is, thanks for your suggestions though.

Blogger Chandan Pitta said...
Ah, now that you mention FPE I remember one more hack I had to do (but forgot to mention in the blog), so I am assuming you are in gentoo. If so then you have to "emerge media-libs/alsa-lib" *with* the debug flag. So added this line
media-libs/alsa-lib debug
to /etc/portage/package.use
and re-emerged media-libs/alsa-lib. That should fix your FPE problem. Let me know how it went