Merging PDF documents in Ubuntu

written by Domen Kožar, on Sep 7, 2009 1:42:00 AM.

Just a note to myself, easiest way to merge .pdf files:
$ sudo aptitude install pdftk
$ pdftk 1.pdf 2.pdf output 12.pdf

Sziget festival 2009

written by Domen Kožar, on Aug 16, 2009 3:26:00 PM.

Rather spontaneous than planned, I decided to visit Sziget festival with 3 old friends. We met one day before the festival began, drinking coffee in Che bar. We chatted about money and food that we need to bring, where do we meet for departure and what is Sziget really all about. I had no idea what bands preform there, or even how to survive on Sziget.

1st day - Traveling

My P1i failed again, leaving me in bed until friends called at 9:20 asking where the hell am I. It took me 3min to pack; and few more to brush teeth, update some website (which was urgent exactly the same moment I wanted to go) and kiss girlfriend goodbye.

Before going on trip, we went to buy about 40 beers to Hofer and at 10:15 we were already on road from Ljubljana (Slovenia) to Budapest (Hungary), which is about 450km apart. The trip was nothing special, we were chased by police on highway, but roundabout saved us. Čeh was nice enough to drive in both directions, so the others were able to open a beer on the way. When we arrived to Obudai island, official parking was already full. They recommended some others (like 10€ per night), but we parked in front of some old socialistic buildings and it turned out to be pretty secure;) We had some concerns about smuggling beers inside the festival, turns out it was pretty easy;)

park the car, drink beer!

Organization of the festival already shined at very beginning, you could really see they have everything thought through. No wonder, the festival has been performed already for last 15 years. Anyway, while we were setting up our tents, Čeh already went to Nouvelle Vague on main stage. Sara, Douč and me joined him later, when they almost finished. SKA-P followed; even though I don't really listen to SKA, mosh pit and crowd surfing were amazing. We split up during mosh pit and I went walking around, with main goal to find some good music to fit my taste.

SKA-P mosh pitting and crowd surfing

This is when I actually first started to feel "the good spirits" for Sziget festival. Near to our tent, I found Budapest Jazz Club stage and listened to Tzumo Electronic Dreams. They really made my heart dance, it was like an entrance to all this festival music. At Jazz stage, one could buy burgers and I decided to go for chicken. After Tzumo, Free Style Chamber Orchestra started to groove some smooth, easy jazz. I decided to go check Blues Kocsma and to my surprise, The Blues B.R.Others Show were on stage trying to give their best music after the two famous brothers.

Čeh came to Kocsma and picked me up, we ran together to 38A stage, listen to the famous White Lies. It's not my type of music, but it was a pretty cozy concert. Light effects were amazing, but maybe the vocals were too low (at least in 4th row). Some chick really rubbed her Italian boy around penis with her ass, which made Čeh really nervous:D Another crazy guy tried to go mosh pitting on White Lies, but people cooled him down quickly. Before the end of the concert, I had to leave to drink some water (those beers really sucked all out of me). After that I wandered around, don't really remember anything interesting that happened:D First day successful!

2nd day - Total Knockout

Woke up at 11:00 and decided to go to Budapest for a "morning" coffee. This was my second time in Budapest (I spent New Year's Eve there, I think in 2003), it is a wonderful city. A little bit older and dirty than Ljubljana, but still very rich. We walked around for quite some time, searching for a "cheap bar". The thing is, we judged them by outlook, not by prices. Ended in bar where coffee was about 2.5€ and frape was only 3€. Noone took coffee;)

After the Budapest tour, we cooked some good Spaghetti with goulash and tomato sauce. At about 16:00 we went for walk around the island, we haven't seen the East part yet. Too bad we arrived at Blues Kocsma at 18:00 when Jimi Hendrix Memorial Band started. Thanks to all the folks that gathered around for alarming us:)

jimmy hendrix memorial band, sziget 2009

I wasn't expecting anything, since Pavle Kavec really gives best spirit to Jimmy Hendrix as far as I've seen. JHM surprised me, the lead guitarist played licks by Hendrix and didn't try to squeeze his own style. Take it as good or bad, people really enjoyed the show. As you can see, people got on stage, kissed the artists, shake'd the ass few times and dove into crowd surfing. I've been to many Jimmy Hendrix tributes, but this one really flowed with energy. Thanks JHM, keep up the good work! (missed Stone Free though;)

Short pause, followed by long, full of energy night. We went to Jack Daniels bar, where one could buy Jack with Cola and got an extra 100 point ticket for Blackjack. The dealers were nicely dressed girls, so tables were pretty full of bad guys. For 1100 points one could get Jack Daniels T-Shirt, I got one while Čeh got three of them. It wasn't that hard after all. The bar had commercial rock music, which was pretty good to relax. People inside got pretty crazy on some songs, so some security guys came closer. Nevertheless, nothing happened. I haven't seen a single blip of anger during 4 days in Sziget.

3rd day - Chill out

On Friday, it started raining just after we woke up. Festival organisation shined even in such circumstances, one could see people running around in raincoats. Čeh, Sara, Douč and me decided to play card game called "Enka" followed by big, fatass New York hotdog. At about 16:00 rain stopped, and we decided to change clothes and split up, each to his own stage.

Again, I went to the Blues stage and listened to New Orleans blues Jambalaya. With Zoltan Nemes on piano/vocals, the blues just got somehow jumpy and soulful. I really liked it, great music to drink a few beers;) Then again, to my surprise, came a band that preformed Jimmy Hendrix, called Póka Egon Experience (sorry I cannot find any info about them on internet). They really had somewhat special attitude against Hendrix and gypsy music. Something special, but solos from lead guitarists really made my heart hurt. Shredding on Hendrix's rythmical and melodical solos is really a great way to take everything from his music.

pretty girls, sziget 2009

Afternoon. We went to eat some chinese noodles and visited Ability Park (which was really relaxing). Main goal is to finish the contest that the toy offers. Most of the times you have to get ball on the other side of the toy. After as-the-spanish-call-it siesta I went to Budapest Jazz Orchestra. Few jazzy songs to relax, and at 23:00 it was time for Jack Daniel's bar, again. Blackjack, rock music, crazy people and Jack Daniels. Night over.

Last day & About Festival

Festival was just awsome. I really liked the idea and challange to collect all different sorts of T-Shirts. WCs were pretty standard, cleaned every day. They even had bigger ones with mirror and AC/DC. You could even get showered by bikini girls and got free swimming pants.

sziget 2009 bikini girls axe shower

Food choice was more or less anything Europe and Asia know. One could eat Hebrew, Chinese, Japan, Greek, Churros, New York Hotdogs, kebab, gyros, mini-donuts, etc.

i want to have sex now, sziget 2009

Čeh also got this crazy idea, called "Tent raping". Girl would bond red headscarf to a tent, which would mean she's all in for action! Well, laughter was far from being absent in Sziget.

Anyway, we got up ta 8am, packed everything including tents, bought 40 cans of beer for 0.4€ and started driving back. We got some problems finding the right way out of Budapest, but in the meantime I bought postcard for my girlfriend Iris (she collects them). I spent about 200€ in 4 days, which is fair enough for that amount of phun;) I'm very sad I forgot to take my SX10IS PowerShot Canon, but hey, one year is quickly around;) May the force be with you!

PS: one free beer to first one that finds my face in this picture.

Ideas for a week project

written by Domen Kožar, on Aug 2, 2009 11:33:00 PM.

On 9th of August I return from vacations and I decided to research on some new area in the week after. Currently, there are two ideas, both easy-to-implement:

1. Ping my host every 5 minutes and send me email if down
I know there are many websites which touch this topic but; Most of them are limited and need subscription. I want something basic, to do just what I want.

WHY:
- learn basic Google App Engine development
- GAE provides reliable hosting with strong uptimes

2. List of websites mapped to tagged keywords
I have not found a simple, yet populated website that would provide list of websites based on some tag/keyword. Example: search for a "subtitle" tag and it would yield all websites where you can download subtitles for movies.

WHY:
- first time get known with MongoDB and it's usage
- learn how to estimate project timeline management with Pylons

What's your opinion? Out in the Web World, is there something exactly what I'm trying to do?

Advanced FormEncode usage

written by Domen Kožar, on Jul 22, 2009 2:11:00 PM.

I've been working on advanced HTML form that uses some Mootools (JavaScript framework) love. In most of my projects I use FormEncode in combination with FormAlchemy. Although both libraries have very different philosophies and areas, they work great together with some patches. I want to share schema because I spent quite some time putting it together - might save you some trobule.

This was the first time I used variabledecode module, and found best reference in this blog post.

Rendered form looks something like this, and if you choose another option in dropdown menu, it rearanges to something like this.

As you can see, AJAX removes some fields, others are rearranged. First form accepts many Name/Surname entries, the second one only gets one pair. I wanted to write a FA schema that can validate both cases and prepares my resulting dictionary to be directly inserted to SQLAlchemy model.
class ApplyForm(formencode.Schema):
    allow_extra_fields = True
    filter_extra_fields = True

    billIsNaturalPerson = validators.StringBool(not_empty=True)

    billEmail = validators.Email(not_empty=True)
    billAddress = validators.UnicodeString(not_empty=True)
    billPost = validators.UnicodeString(not_empty=True)
    billPhone = InternationalPhoneNumber(default_cc=386, not_empty=True)

    billFirm = validators.UnicodeString(if_missing=None)
    billTaxNumber = SlovenianTaxNumber(if_missing=None)
    billIDNumber = validators.Int(if_missing=None)
    billContactPerson = validators.UnicodeString(if_missing=None)

    billIsDDV = validators.Bool(not_empty=True, if_missing=None)
    billIsProformaInvoice = validators.Bool(not_empty=True)
    billIsSIR = validators.Bool(not_empty=True)
    billIsDinner = validators.Bool(not_empty=True)

    relParticipants = formencode.ForEach(ParticipantForm)
    relDaysAttending = validators.DictConverter({
        '1': [model.meta.Session()\
            .query(model.PortorozDay)\
            .filter(model.PortorozDay.plPDayDate == date(2009, 11, 18)).one()], # wed
        '2': [model.meta.Session()\
            .query(model.PortorozDay)\
            .filter(model.PortorozDay.plPDayDate == date(2009, 11, 19)).one()], # thu
        '3': model.meta.Session()\
            .query(model.PortorozDay).all() , # both
    }, not_empty=True, hideDict=True)

    pre_validators = [NestedVariables()]
    chained_validators = [
        validators.RequireIfPresent('billFirm', missing='billIsNaturalPerson'),
        validators.RequireIfPresent('billTaxNumber', missing='billIsNaturalPerson'),
        validators.RequireIfPresent('billIDNumber', missing='billIsNaturalPerson'),
        validators.RequireIfPresent('billContactPerson', missing='billIsNaturalPerson'),
    ]

class ParticipantForm(formencode.Schema):
    allow_extra_fields = True
    filter_extra_fields = False

    parName = validators.UnicodeString(if_missing=None)
    parSurname = validators.UnicodeString(if_missing=None)

    chained_validators = [
        validators.RequireIfPresent('parName', present='parSurname'),
        validators.RequireIfPresent('parSurname', present='parName'),
        MapToModel(model.PortorozParticipant),
    ]
First of all, it's all about the state of billIsNaturalPerson. This is the key boolean that separates both forms. As you can see in chained_validators, if it's value is False, it will require some additional fields. Fields with if_missing=None are required only in first form. Javascript removes input elements from DOM, so SQLAlchemy model is populated with None values.

relDaysAttending is a SQLAlchemy relation and in form terminology, a radioset. Values are mapped with help of DictConverter validator to model instances, which results in correctly populated ForeignKeys.

relParticipants is also a SQLAlchemy relation that lists name/surname pairs. As you can see, it's validated with help of variabledecode algorithm and validator that corresponds to that is NestedVariables. Because NestedVariables validators lies in pre_validators attribute, when validator occurs, values are already mapped to list/dictionary. Input names look something like this:
<div>
    <input type="text" name="relParticipants-1.parName" value="John" />
    <input type="text" name="relParticipants-1.parSurname" value="Smih" />
</div>
<div>
    <input type="text" name="relParticipants-2.parName" value="..." />
...
Which maps to list of dicts:
{'relParticipants': [
   {'parName': 'John', 'parSurname': 'Smith'},
   ...
]}
ForEach validator later maps relParticipant list to ParticipantForm schema. This schema is pretty basic, chained validators are used to require second field (if one is supplied) and map resulting dict to model instance. MapToModel implementation is very simple:
class MapToModel(validators.FormValidator):
    __unpackargs__ = ('model',)
    model = None

    def _to_python(self, value_dict, state):
        return self.model(**value_dict)
Example of parsed POST data, mapped to be ready for SQLAlchemy model:
18:24:07,349 DEBUG [project.controllers.portoroz] Form input:
{'billAddress': u'Foo street 12',
 'billContactPerson': u'John Smith',
 'billEmail': u'john.smith@gmail.com',  
 'billFirm': u'Firm',
 'billIDNumber': 1337,
 'billIsDDV': None,
 'billIsDinner': False,
 'billIsNaturalPerson': False,
 'billIsProformaInvoice': True,
 'billIsSIR': False,
 'billPhone': '+386-11-444489',
 'billPost': u'1000 Ljubljana',
 'billTaxNumber': 40952649,
 'relDaysAttending': [<PortorozDay 2009-11-18>, <PortorozDay 2009-11-19>],
 'relParticipants': [<PortorozParticipant(None) parName=u'foo', parSurname=u'bar'>,
                     <PortorozParticipant(None) parName=u'hello', parSurname=u'kitty'>,
                     ]
}
Problems started when I tried htmlfill and it's error rendering. First issue was that I needed to reverse variabledecode algorithm, this can be achieved with following e.unpack_errors call:
        try:
            c.form_result = ApplyForm().to_python(request.POST)
        except formencode.validators.Invalid, e:
            return formencode.htmlfill.render(
                self.apply_action(),
                defaults=request.POST,
                errors=e.unpack_errors(formencode.variabledecode.variable_encode))
Second issue was if user added lots of Participants, they were not correctly populated in form inputs because by default only one was being rendered and additional ones are inserted by javascript. I decided to take most ugly but simple approach: by default render 30 fields and onDOMReady, remove the all the empty fields except one.

There are details I left out of this post, since main concern was FormEncode usage. I hope you learned something, cheers!

Persuading Paste HTTP server to use correct REMOTE_ADDR

written by Domen Kožar, on Jun 27, 2009 12:58:00 AM.

Configuring reverse proxy headers was on bottom of my TODO list because I never felt it was important for my applications. Today I tried to fuddle with nginx to pass those headers but it gave me a headache.

For sake of reference and because I'm sure many others tripped on this one, here is what I figured out from mailing lists and source code (please inform me of a better, straight forward way if there is one):

Add to your deployment.ini file:
[app:main]
# ...
filter-with = proxy-headers

[filter:proxy-headers]
use = egg:PasteDeploy#prefix
Prefix middleware actually extracts proxy headers information and passes results forward. Now, nginx config:
    location / {
        proxy_pass http://localhost:5000;

        proxy_set_header X_FORWARDED_SERVER $server_name;
        proxy_set_header X_FORWARDED_FOR $proxy_add_x_forwarded_for;
        proxy_set_header X_FORWARDED_HOST $proxy_host;

    }

Joining strings in Python

written by Domen Kožar, on Jun 11, 2009 2:58:00 PM.

There is already a lot of good material to read about performance. Therefore, it is best to use the following method:
    >>> print ' '.join(['The', 'fox', 'jumped', 'over', 'the', 'dog.'])
    The fox jumped over the dog.
Perfect. We have very fast string contatenation that even allows us to choose separator. Now, here is the problem:
    >>> print ', '.join(['apples', 'oranges', '', 'cocos'])
    apples, oranges, , cocos
So, if we dynamically generate string and we have a separator for a reason, we do not want this to happen. This is basically the idea:
    >>> print ', '.join(filter(None, ['apples', 'oranges', '', 'cocos']))
    apples, oranges, cocos
As you can see, filter(None, iterable) returns list of elements that evalue to True. This way we have somewhat a safe string contatenation. Real world example:
class StringBuffer:

	def __init__(self, sep=''):
		self.sep = sep
		self.output = list()

	def write(self, content):
		self.output.extend(content)

	def getvalue(self):
		return self.sep.join(filter(None, self.output))


>>> sb = StringBuffer(', ')
>>> sb.write(['apples'])
>>> sb.write(['', None])
>>> sb.write(['oranges', 'cocos'])
>>> print sb.getvalue()
apples, oranges, cocos

Mocking logging module for unittests

written by Domen Kožar, on Mar 4, 2009 6:07:00 PM.

Today I was writing tests for upcoming application I'm writing (more information will come later) and I needed some way to redirect logging to some kind of storage that could be easily asserted to values in tests. First I tried with monkey patching it, but that wasn't the solution since all logging setup is done before pylons tests come to play, so somebody in IRC pointed me toward writing a handler. I thought to do so in the first place, but were too stubborn;) Here is the handler code:
class ListHandler(logging.Handler):

    debug = []
    warning = []
    info = []
    error = []

    def emit(self, record):
        getattr(self.__class__, record.levelname.lower()).append(record.getMessage())

    @classmethod
    def reset(cls):
        for attr in dir(cls):
            if isinstance(getattr(cls, attr), list):
                setattr(cls, attr, [])
So basically, all records are stored in handlers class(!), which is restored to zero on every tearDown() call. Later in the code, I can do stuff like:
    def test_foobar(self):
        some_function_that_outputs_logging()
        self.failUnless('Invalid status' in  ListHandler.warning)