Working with Big Patchsets

In the past, whenever there was a need for me to test or review huge patchsets, I'd generate individual patches by saving the mails containing them. This can get cumbersome, especially if the patchset is huge and if you have to try out multiple versions of the patches. There is however an alternative available. It is to obtain the mbox file pertaining to the entire patchset. The mbox file for kernel patches can be obtained from patchworks. The home page displays a list of projects that are hosted. Below are a few steps to get started -

1. Create a login on patchworks
   
2. Once in your account, select the project of interest from the home page
3. Using the filters, search for the particular patchset
4. Select the individual mails of the patchset and create a new 'bundle' from the interface at the bottom of the page. 'bundles' are a group of related
   patches. Name the bundle appropriately
   
5. You can go to all your bundles by clicking on the 'bundles' link at the top right corner of any page
6. The bundles interface allows you to manage your bundles, with support to download it. Bundles are nothing but all the mails in a mbox format
7. You can then open the mbox with either your mail client or apply it directly to your tree with patch management tools that allow importing mbox files. I use mutt to read the mbox file and stgit to import the patches

Avoid Kmail Editor And Use Vim Editor Instead

Kmail has bad editor when it comes to sending patches inlined. More often the message gets mangled (word wrapped). To overcome this problem Kmail allows the use of external editor.

1. Settings->Configure Mail and Click on Composer
2. Click on "Use External Editor" and specify the editor as "xterm -e vim -f %f"
3. Click Apply.

Open the composer and type in mail id and subject and click on the body. When you start typing a new editor window opens up which is vim. Type in your message and type ":wq". Thats it.

For people intending to send patches, use :vsplit and use visual method to highlight and 'y' to yank the highlighted message. Close the vsplit and type "p" to paste it.

Convert VCD T0 AVI (Xvid)

A simple way to convert old vcds to avi(xvid) and not lose too much video quality is by using the tool mencoder.

Download and Install mencoder:
#sudo apt-get install mencoder

Start Conversion:
#mencoder vcd://2 -o movie.avi -oac copy -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=2000

Where vcd://2 points to the title you wish to decode. Most vcds have 2 to 3 titles and the biggest one being the movie. Choose the biggest title of vcd for the decode. If you are unsure what to choose, i suggest trial and error starting from 1 to any number till mencoder throws out error of not finding the title.

Rip mp3 from cue/flac!

Here is a simple way to rip mp3 from flac files. There are few softwares that need to be installed for to work.

Download and Install Sound Converter For Linux. It can be downloaded from here.

Once Installed you can start up the application from the multimedia panel in the start menu.

Install additional packages
#sudo apt-get install flac mp3splt

Step 1: Convert the flac file to mp3 using Sound Converter. This process is quite simple. Start the sound converter and specify the flac file you wish to convert and press "Convert". This converts the flac file to one single mp3 file. But we are aware of multiple songs in that mp3 file.

Step 2: Split the single mp3 files into multiple mp3 tracks. This can be done using the mp3splt utility we just installed.
#mp3splt -f -c Album.cue Album.mp3
Assuming that Album.mp3 is the file produced by the Sound Converter. "cue" files are generally provided with flac files for this very purpose. If you do not have the cue file, you have to use some mp3 splitter to achieve the same.

Step 3: The new mp3 files might have issues with reading mp3 duration of these files. If that happens then a simple tool called vbrfix will fix the problem.
Install vbrfix
#sudo apt-get install vbrfix

To fix a mp3 file using vbrfix
#vbrfix myfile.mp3 myfile_fix.mp3

The myfile_fix.mp3 would have the duration problem fixed.

Real-Time Linux Kernel Scheduler

My article explaining the Real-Time Linux kernel scheduler is finally available for general reading on the Linux Journal site. You can read it here. Any feedback on the article are most welcome !

Dumping kernel page tables

Sometimes when debugging kernel issues, you might come across kernel addresses that you would find very difficult to map to a particular section in the kernel, ie, vmalloc, vmemmap, low/high kernel mapping, kernel text, etc. On x86, Arjan van de Ven has written an interface that provides a dump of the kernel page tables which gives information on the various memory areas in the kernel.

# cat /debug/kernel_page_tables
---[ User Space ]---
0x0000000000000000-0xffff800000000000 16777088T pgd
---[ Kernel Space ]---
0xffff800000000000-0xffff880000000000 8T pgd
---[ Low Kernel Mapping ]---
0xffff880000000000-0xffff880000200000 2M RW GLB x pte
0xffff880000200000-0xffff880040000000 1022M RW PSE GLB x pmd
0xffff880040000000-0xffff8800cfe00000 2302M RW PSE GLB NX pmd
...
---[ vmalloc() Area ]---
0xffffc20000000000-0xffffc20000001000 4K RW PCD GLB NX pte
0xffffc20000001000-0xffffc20000004000 12K pte
0xffffc20000004000-0xffffc20000005000 4K RW PCD GLB NX pte
0xffffc20000005000-0xffffc20000008000 12K pte
0xffffc20000008000-0xffffc2000000d000 20K RW PCD GLB NX pte
0xffffc2000000d000-0xffffc20000010000 12K pte
0xffffc20000010000-0xffffc20000011000 4K RW PCD GLB NX pte
....
---[ Vmemmap ]---
0xffffe20000000000-0xffffe20007c00000 124M RW PSE GLB NX pmd
0xffffe20007c00000-0xffffe20040000000 900M pmd
0xffffe20040000000-0xffffe28000000000 511G pud
0xffffe28000000000-0xffffff8000000000 29T pgd
0xffffff8000000000-0xffffffff80000000 510G pud
---[ High Kernel Mapping ]---
0xffffffff80000000-0xffffffff80200000 2M pmd
0xffffffff80200000-0xffffffff80a00000 8M RW PSE GLB x pmd
0xffffffff80a00000-0xffffffffa0000000 502M pmd
---[ Modules ]---
0xffffffffa0000000-0xffffffffa000a000 40K RW GLB x pte
0xffffffffa000a000-0xffffffffa000f000 20K pte
0xffffffffa000f000-0xffffffffa0016000 28K RW GLB x pte
0xffffffffa0016000-0xffffffffa001b000 20K pte
....
...

Understanding the above output:

o First field indicates the address range of a particular type of area (for example, user space, vmalloc area, kernel space, etc)
o The second field indicates the size of the address range in K,M,G,T units
o The fields following the size of the range have the following meaning:
USR - whether the page being mapped is a user page or not
RW - whether the page is read/write. If not RW, the output would be 'ro' to indicate a read-only page
PCD - Page Cache Disabled - maps a page with caching disabled
PWT - page with Page Write-Through set
PSE - Extended paging enabled - allows large linear contiguous address ranges to be mapped
GLB - Page Global flag - The global flag is set for a page that is frequently used and prevents it from being flushed from the TLB
NX - Page is non-executable, else marked as 'x'
o The last entry indicates the particular level of the page table - pgd, pud, pmd or pte that the region corresponds to

Enable the CONFIG_X86_PTDUMP configuration option, along with enabling debugfs. The corresponding kernel code for the interface can be found under arch/x86/mm/dump_pagetables.c

Using gdb for debugging kernel modules

This is a very simple and maybe trivial post.. and nothing novel ! But something that has helped me a lot in debugging issues with my kernel modules and kernel panics in general. So, when you get a panic originating in a kernel module, you normally get it in the following format:

BUG: unable to handle kernel NULL pointer dereference at 0000000000000050
IP: [<ffffffff8023262c>] mmput+0x11/0xb0
PGD 22bcf6067 PUD 211839067 PMD 0
Oops: 0002 [#1] SMP
last sysfs file: /sys/block/sdb/size
CPU 6
Modules linked in: <a routine in my module> ehci_hcd uhci_hcd [last unloaded: <my module>]
Pid: 26830, comm: <my kernel thread> Not tainted 2.6.28-anki #13
RIP: 0010:[<ffffffff8023262c>] [<ffffffff8023262c>] mmput+0x11/0xb0
RSP: 0000:ffff8802279fbed0 EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000045
RDX: ffff88022e150300 RSI: 0000000000000282 RDI: 0000000000000000
RBP: ffff8802279fbee0 R08: 0000000000000018 R09: ffff88022e150300
R10: ffff88022e150300 R11: 00007fffe6dfefff R12: ffff88022e52f860
R13: ffff88022e150300 R14: ffff88022d99aed0 R15: ffff88022d99b110
FS: 0000000000000000(0000) GS:ffff88022f818280(0000) knlGS:0000000000000000
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000000000000050 CR3: 0000000211829000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process memref (pid: 26830, threadinfo ffff8802279fa000, task
ffff88022d4ef1b0)
Stack:
ffff88022e150300 0000000000000000 ffff8802279fbf20 ffffffffa003314b
ffff8802279fbf20 ffff88022d99aed0 ffffffffa00330f2 ffff880210cf1c68
ffff88022f855ee0 ffff88022f855f00 ffff8802279fbf40 ffffffff802468e3
Call Trace:
[<ffffffffa003314b>] <routine in my module>+0x59/0xb2 [<my kernel thread>]
[<ffffffffa00330f2>] ? <my modulee>+0x0/0xb2 [<my kernel thread>]
[<ffffffff802468e3>] kthread+0x49/0x76
[<ffffffff8020cce9>] child_rip+0xa/0x11
[<ffffffff802277a5>] ? dequeue_task+0xbf/0xca
[<ffffffff8024689a>] ? kthread+0x0/0x76
[<ffffffff8020ccdf>] ? child_rip+0x0/0x11
Code: df e8 88 c7 fd ff 48 8b 3d ba 28 6b 00 48 89 de e8 b1 af 05 00 41 59 5b
c9 c3 55 48 89 e5 53 48 89 fb 48 83 ec 08 e8 6a fa 39 00 <f0> ff 4b 50 0f 94
c0 84 c0 0f 84 8b 00 00 00 48 89 df e8 8b 76
RIP [<ffffffff8023262c>] mmput+0x11/0xb0
RSP <ffff8802279fbed0>
CR2: 0000000000000050

Now, to find out from where exactly in my module the panic happened, I simply use gdb, as below:

#gdb <path to my module.o file from the kernel source root dir> [for example, #gdb drivers/misc/ankita.o]
....
(gdb) list *(the routine)+0x59

The above gives a source listing, pointing to the lines which correspond to 0x59 address. This serves as a good starting point to debug the issue. Hope this helps some folks atleast :-)