display_rotate=3 black screen

[Nuker]

hdmi_mode=82 - allmost good, but since i rotate screen 270 degrees, i need 1080x1920 instead of 1920x1080
hdmi_mode=36 - not full screen, also resolution to low.
in the RPi config page, there is a link to thread http://www.raspberrypi.org/phpBB3/viewtopic.php?f=29&t=24679 that with
hdmi_cvt <width> <height> <framerate> <aspect> <margins> <interlace> <rb>

you can set custom resolution, which is what i want. however, on xbian it doesn't seem to have any efect.

http://pastebin.com/AtVWwdrR - dmesg doesn't show any bcm2708 errors.