Archive

Archive for the ‘General’ Category

Update table from select.

November 18th, 2010 No comments

Here’s a small example how you can update a table from a select statement.

UPDATE Contact
SET Contact.Surname = x.Surname
FROM (
	SELECT Id, SUBSTRING(Surname, Len(Insertion) + 2, LEN(Surname) - Len(Insertion)) AS Surname
	FROM Contact
	WHERE Insertion IS NOT NULL
	AND Insertion <> ''
	AND CHARINDEX(Insertion + ' ', Surname) = 1) x
WHERE Contact.Id = x.Id;

Using a join is also possible.

UPDATE c
SET c.Body = x.Body, c.Summary = x.Summary
FROM Content c
JOIN PageContent x ON c.ID = x.ID;
Categories: General Tags: , ,

Converting media for your PS3

August 26th, 2010 No comments

According to the PS3 site the following media can be played:

The following types of files can be played under (Video).

    * Memory Stick Video Format
    - MPEG-4 SP (AAC LC)
    - H.264/MPEG-4 AVC High Profile (AAC LC)
    - MPEG-2 TS(H.264/MPEG-4 AVC, AAC LC)
    * MP4 file format
    - H.264/MPEG-4 AVC High Profile (AAC LC)
    * MPEG-1 (MPEG Audio Layer 2)
    * MPEG-2 PS (MPEG2 Audio Layer 2, AAC LC, AC3(Dolby Digital), LPCM)
    * MPEG-2 TS(MPEG2 Audio Layer 2, AC3(Dolby Digital), AAC LC)
    * MPEG-2 TS(H.264/MPEG-4 AVC, AAC LC)
    * AVI
    - Motion JPEG (Linear PCM)
    - Motion JPEG (μ-Law)
    * AVCHD (.m2ts / .mts)
    * DivX
    * WMV
    - VC-1(WMA Standard V2)

Supported audio formats:

    * Memory Stick Audio Format(ATRAC)
    * MP3
    * - MPEG-1/2 Audio Layer3
    * - MP3 Surround
    * MP4(MPEG-4 AAC)
    * WAVE(Linear PCM)
    * WMA

I tested several encodings on my Quad-core Intel (Q6600). I got the best results encoding with the MP4 format as all four cores were used at 100%. This also resulted in smaller file sizes.
I am using VLC to do the conversion although other programs (ffmpeg, mencoder) might also work mighty fine.

The goal I had was to transcode movies add subtitles and keep file sizes as small as possible so we can put them on an external usb storage device for playback.

We will use VLC from the command-line. The following commands are useful. Have a look at the man pages for more information. VLC command-line pages

This is the command I ended up with:

vlc "movie.avi" --sub-file="movie.srt" --sout=#transcode{vcodec=h264,acodec=mp4a,vb=0,ab=192,channels=2,soverlay,audio-sync}:standard{access=file,mux=mp4,dst="outputfile.mp4"} vlc://quit

You can add “-I dummy” if you don’t want/need a window to pop up. I like it because it shows where you are in the transcoding process.
The soverlay is used for the subtitles. I limit the audio channels to 2 because I don’t have any surround sound setup. Change it to whatever you need.

I also happen to have a full hd camera which has a sensor in the format 1440×1080. These pixels have an aspect ratio 4:3 (PAR) which results in getting movies with 1920×1080 resolution. (Full HD) To be able to play these movies on my PS3 I convert them with FFMPEG. I have built FFMPEG from source on my linux machine because I need libx264 to encode the videos. The command I then use to get PS3 compatible movies is:

ffmpeg -i Moviename.MOV -sameq -vcodec libx264 -vpre slow -crf 22 -acodec libfaac -ab 128k Moviename.mp4

Modify the command as needed. Above line gives me good quality and decent file sizes.

The paperless office

August 17th, 2008 No comments

Well here we are once again. This time to conquer the pile of paper laying besides, on and under your desk. My girlfriend went nuts by the sheer load of paper laying around everywhere. I’m kind of messy but when I start to organize things I’m kind of a perfectionist. So here I went and put all the papers in one big pile.

After going through the pile I realized that there were a lot of papers worth trowing away but at the same time we’re worth keeping. A dilemma. So I had a good cold beer and started thinking about the situation. The beer worked like oil on the brains and I came up with a good solution. ‘Let’s scan this pile of toiletpaper!’ and so I provided my HP 7310 with some juice and put all the paperwork on the automatic paper input of the device. I inserted an empty SD-Card and started gaming while the HP started to do what it was made for.

After a few hours ( well actually the scanner was long done before that… plz don’t tell my girlfriend 😉 ) the scans were complete and I put the files on my debian server. So far so good. Now at least I had a backup of all the paper versions. After that I was rather satisfied and threw away a large pile of paper I no longer needed as I had a digital backup now.

But a real geek doesn’t stop here. What I wanted next was to be able to search through my digital paperwork fast and get the papers that I need. So I had a look and found that there is a great open source ocr package called tesseract. I downloaded some packages and started trying out some things. I found out that it was only capable of handling tiff images at this point and that it was best to avoid color in the images. To get the required images I installed ImageMagick to do the converting from jpg to grayscale uncompressed tiffs.

So far so good. This will result in a txt file containing the text in the file. Pretty neat. Now I can use grep to look for any string matches and then open the matching jpg. Easy as 1,2,3. 🙂

After this I created the following script that will parse any new images automatically with the ocr software.

#!/bin/bash
 
basedir="/share/downloads/Scans"
 
# Parses a scan if not already processed
# $1 = JPG file
parse_scan(){
        jpg_file=$1
        base_name=${jpg_file:0:(${#jpg_file}-4)}
        tif_file=$base_name.TIF
        txt_file=$base_name.txt
 
        # If tiff file does not exist, use imagemagick to convert
        if [ ! -e $txt_file ]; then
                echo "Converting: $jpg_file into $tif_file"
 
#               echo $base_name
#               echo $tif_file
#               echo $txt_file
 
                # Convert jpg into tiff file
                convert $jpg_file -format tiff -colorspace gray -depth 8 -compress none $tif_file &> /dev/null
 
                # Use tesseract for ocr on tiff file
                tesseract $tif_file $base_name -l nld &> /dev/null
 
                # Remove tiff file
                rm $tif_file &> /dev/null
        fi
}
 
for file in $basedir/*; do
        filename=$file
        length=${#filename}
 
        if echo $filename | grep -q '.jpg$'; then
                # Create TIF filename
                parse_scan $filename
        elif echo $filename | grep -q '.JPG$'; then
                # Create TIF filename
                parse_scan $filename
        fi
done

You can of course make the basedir an argument which can be passed into the shell script. But that is a personal choice. Now you can get rid of your pile of paper too. Let’s start recycling that pile of paper. 🙂

Subversion (SVN) Server Startup Script

February 12th, 2008 2 comments

To be able to start and stop my subversion server nicely I created the following script to startup and shutdown under debian.

#! /bin/sh
# /etc/init.d/svnserve: start and stop svnserve
 
# Exit immediately if a command exits with a nonzero exit status.
set -e
 
# svnserve exists and is executable
test -x /usr/bin/svnserve || exit 0
 
# Directory Where the Repository is located, created with svnadmin create
REPOS_DIR="/opt/svn"
 
# The pid-file
PIDFILE="/var/run/svnserve.pid"
 
case "$1" in
  start)
        echo -n "Starting Subversion (SVN) Server"
        start-stop-daemon --start --quiet \
                --exec /usr/bin/svnserve -- -dr $REPOS_DIR
        # Also tried the following line but it didn't work ok
        #       --make-pidfile --pidfile $PIDFILE
        PID=`pidof svnserve` || true
        echo $PID &gt; $PIDFILE
        echo "."
        ;;
  stop)
        echo -n "Stopping Subversion (SVN) Server"
        start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE
        if [ -f $PIDFILE ]
        then
                rm $PIDFILE
        fi
        echo "."
        ;;
 
  *)
        echo "Usage: /etc/init.d/svnserve {start|stop}"
        exit 1
esac
 
exit 0

Place the script in /etc/init.d/svnserve and run the following command:

update-rc.d svnserve defaults

Tomcat behind reverse proxy on Apache

January 23rd, 2008 1 comment

If you have apache installed as your main http server you might want all requests to be handled by apache. But what if you want to have a standalone tomcat install? Well this is still possible by using apache’s mod_proxy. You might need to install it if you are running debian. Do some ‘apt-cache search apache proxy’ abracadbra and install mod_proxy if it is not already installed. You might even need to enable it with ‘a2enmod proxy’.

Edit your configuration (I added it to the sites-available/default) and add something like this:

    # Tomcat Proxy
    RedirectMatch ^/tomcat$ /tomcat/
    ProxyRequests Off
    ProxyVia Off
    ProxyPass /tomcat/ http://localhost:8082/
    ProxyPassReverse /tomcat/ http://localhost:8082/

You also might want add some access rules to make sure noone will be able to abuse your proxy (especially when running a forward proxy).

<proxy *:80>
Order deny,allow
Deny from all
</proxy>
<proxy *:8080>
Order deny,allow
Deny from all
Allow from 127.0.0.1
</proxy>

This should do the trick (after reloading apache of course, /etc/init.d/apache2 reload). The redirectmatch will make sure that users that reach http://serveraddress/tomcat will be redirected to http://serveraddress/tomcat/ because otherwise they would not reach the page.
The proxypass points to /tomcat/ note the trailing slash. If you omit the trailing slash your images will not load correctly. port 8082 is tomcat’s proxy port which will allow proxied connect calls.

You can use the same trick for any other webserver you have running on another port. (webmin, azureus web html, etc)

Setting up an apache2 proxy server

January 15th, 2008 5 comments

note: I found out that you can also use the ssh -D option to have ssh function as a SOCKS server to get similar behaviour without the hassle of configuring apache.

Are you at work behind a big bad evil proxy? Afraid of your privacy? Set up a proxy on your local home server so you can browse safely.

What are we going to use:

  • Putty
  • Apache 2

Is that all we need? Yes that’s all. 😉

Ok let’s set up our apache 2 proxy first. It is a good idea to add some security to your proxy server so not everyone can reach it. You might want to restrict it to the localhost only. This tutorial is based on debian install of apache 2. So hang on and let’s go.

First change the ports apache2 is listening in to. Edit /etc/apache2/ports.conf for this purpose and add the line:

Listen 8080

This will make the apache2 server listen to port 8080. If there is a line that makes apache2 listen on port 443 (https) you might want to disable it. We are going to use putty to connect to this port.

You might need to download mod_proxy for apache2 to be able to use proxying. If it is not already enabled use:

a2enmod proxy
a2enmod proxy_connect
a2enmod proxy_html
a2enmod proxy_ftp

This will enable it. If mod proxy is not yet installed at all use an apt-cache search mod proxy to locate and install it through apt.
the mod proxy_connect is required to be able to handle SSL calls through your proxy.

Create a new entry in /etc/apache/sites-available and name it ‘proxy’ for example. Insert something like the following:

<virtualhost *:8080>
        ServerAdmin webmaster@localhost
 
        ProxyRequests On
        ProxyVia On
 
        #Add ports you want to be able to connect to through your proxy here
        AllowCONNECT 443 563 1863
        #443   = SSL
        #563   = TLS
        #1863  = MSN Messenger
 
        DocumentRoot /var/www/
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
 
        ErrorLog /var/log/apache2/proxy-error.log
        TransferLog /var/log/apache2/proxy-transfer.log
        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel notice
</virtualhost>

Enable this new site by typing:

a2ensite proxy

It would be really wise to limit the access to your forward proxy.
edit the proxy.conf file in /mods-available/proxy.conf.
Add something like this to allow only connections from localhost:

<proxy *:80>
    Order Deny,Allow
    Deny from all
</proxy>
<proxy *:8080>
    Order Deny,Allow
    Deny from all
    Allow from 127.0.0.1
</proxy>

reload the webserver after this by running:

/etc/init.d/apache2 reload

Add 443 to your ssh listen ports by opening /etc/ssh/sshd_config.
Edit it so it reads:

# What ports, IPs and protocols we listen for
Port 22
Port 443

Restart the ssh daemon by calling:

/etc/init.d/ssh restart

Now you are ready to go. You can try your proxy now by using putty to connect to your server and tunnel port 8080 to another port on your local machine. It might also be a good idea to enable zip compression on your connection (Putty:Connection->SSH->Enable Compression) to speed things up a bit.
Now you can use firefox or another app and connect on localhost:[bound putty port] to connect to your proxy.

If you want firefox to do the dns lookups on the remote end you should open your about:config page by typing this in the address bar. Lookup the value:

network.proxy.socks_remote_dns

Set the value to true to do remote dns lookups.

Setting up Rsync

January 14th, 2008 No comments

I was getting fed up with having to synchronize several folders containing photos. So I was looking for a good way to synchronize these images across the network. I first tried to put the images into Subversion but this turned out to be overkill. So that’s why I put my focus on Rsync.

Rsync has support for incremental file exchange. Not that I am going to use it but it still is cool. I will only use Rsync to keep my photo directories synched with the server. So lets install it.

apt-GET install rsync

Edit the rsyncd.conf file in the /etc directory and create some entries here. Example is my photos entry here.

#/etc/rsyncd.conf
#file containing username+passwords IN the form <name>:
<password> the file should be readable ONLY BY USER OR GROUP.
#so chmod 660 OR 600 would be necessary.
secrets file = /etc/rsyncd.secrets
motd file = /etc/rsyncd.motd 
 
#Below are actually defaults, but TO be ON the safe side...
READ ONLY = yes
list = yes
# WITH what USER permissions should rsync handle directories?
uid = nobody
#gid = nobody
#On debian GROUP nobody IS mapped TO nogroup
gid = nogroup
#if stricts mode IS TRUE secrets file should NOT be readable BY ALL. TO disable this turn stricts mode TO off. (built IN FOR cygwin users)
strict modes = TRUE
 
[photos]
comment = Family Photos
path = /share/photos
auth users = wytze
READ ONLY = no
hosts allow = 192.168.*
hosts deny = *
list = FALSE
</password></name>

Start the rsync daemon:

rsync --daemon

Now you can start synchronizing. Synchronizing works like copying files with scp. So it’s fairly straightforward.

To retrieve a list of entries if listing is enabled:

rsync -avz wytze@debian::

To synchronize files from server:

rsync -avz wytze@debian::photos /opt/my-local-photos

To synchronize files to the server:

rsync -avz /opt/my-local-photos/ wytze@debian::photos

Please mind the trailing slashes. Try out the difference with and without the trailing slash. Pretty straightforward there.

Categories: General Tags:

Sending commands to detached screens

November 5th, 2007 No comments

For running Azureus headless on my linux machine I use screen so it runs in a separate screen.

The command to do this is the following:

screen -d -m -S azureus su -p azureus -c "/opt/azureus/azureus.sh"

This will create a detached screen with the name “azureus” which will run the command “su -p azureus -c “/opt/azureus/azureus.sh”

To send a command to this detached screen we can use the following command structure: (Which I use to tidily close the process)

screen -S azureus -p 0 -X eval 'stuff quit\015'

-p 0 tells screen to use window 0 on the specified screen. Using the eval function makes sure the \015 char will be evaluated back into an enter character. The stuff command is used to fill the input buffer of the screen program.

After this the command “quit” is sent to the azureus client which will cause it to quit and close the attached screen.

Categories: General Tags: , , ,

MoBlock and Hamachi

October 12th, 2007 No comments

After installing moblock (a p2p guardian program) I found that my hamachi was no longer working. 🙁

To fix this I had to modify the moblock.conf file in the /etc/moblock directory.
I did so by specifying a network address with a netmask.

Hamachi’s ip-range starts with a 5. So the address would be 5.0.0.0
The netmask is 255.0.0.0 to allow all hamachi ip’s to connect. Because 255.0.0.0 means we have 8 bits on 1. The line becomes: 5.0.0.0/8

Add this line to the IP_TCP_IN line for example and restart moblock with moblock-control restart. After I did this I was able to reconnect with my linux server from hamachi. Hurray!

[update]Ok, maybe you also want to add LOGMEIN to your ignore block list. :S
Do so by changing the IP_REMOVE line into IP_REMOVE=”LOGMEIN”.
Do a moblock-control reload after this to reload the new configuration[/update]

Categories: Coding, General Tags: