http://hacking.certifiedsecure.com/showthread.php?63-Introductie-op-het-ELF-binary-format

It deals with the structure inside an ELF file, the use of sections in binaries, and goes to show how one can disssect the /bin/ls binary using the tools readelf and objdump.

Tweet This Post

Tweet This Post

I didn't bother with the project after placing an initial version on google code, but this week I thought I might pick up development again. It occurred to me that few people writing AMF services will think through the security aspects of their service. You don't make service calls by hand, so it's easy for a developer to assume that only his app will be making calls. Wrong.

At least, that would be nice. As it turns out, the tool (which I've spent only 2 days on developing) is already being recommended for pentesting of AMF services. Which attests to the complete lack of tools, but still is nice

I'll be picking up development again and adding some fuzzing functionality. If you have any feedback, please leave it here or at the google code project!

Tweet This Post

When using the ImageMagick "convert" command, my thumbnails were way to large. When resizing a large image to create a thumbnail, the thumbnail would be 41k while it should be around 4k. The input image was 1600x1200, 300DPI, 594k.

After searching for a while, I found that you shouldn't use the -scale or -resize option, but the -thumbnail option. This strips profile data from the image. Apparently, the image contains profile data which is left intact when resizing/scaling, but can be stripped with the thumbnail option.

Tweet This Post

First, you need to install the Flex 3.3 SDK. If you're running an older Flex Builder, it will have 3.1 or 3.2 included. You can download the Flex SDK from Adobe here: http://www.adobe.com/cfusion/entitlement/index.cfm?e=flex3sdk. Be sure to also download the file labelled "ADOBE FLEX 3.3 DATA VISUALIZATION COMPONENTS FOR FLEX BUILDER".

After downloading, unzip the SDK somewhere and remember the location. Then, in Flex Builder, go to the preferences -> installed Flex SDKs -> Add. Enter the location of the SDK you unzipped.

Some of your apps might use classes that are no longer included with the SDK but bundled separately in the Flex Data Visualization Components you just downloaded. You will know this when you get an error like the following:

1017: The definition of base class HierarchicalData was not found.

Unzip the data visualization components zip, and move the swc and other files in it to the Flex 3 SDK dir in the fashion described in the readme.

Now, you need to change the namespace of the AIR app to 1.5. This is similar to setting the target Flash Player version in a Flex project. It is done by editing the -app.xml file belonging to the project and changing the first line to:

<application xmlns="http://ns.adobe.com/air/application/1.5">

If you get the following error: "error while loading initial content" when running an AIR app, you have upgraded your SDKs to 3.3 but not changed the namespace to 1.5.

Well, that's it, not so bad after all.

Tweet This Post

I suggested to the commissioner of the project (Axis.fm) that we release the tool under GNU/GPL, and they agreed. So, the tool can be found now on http://code.google.com/p/pinta.

What the tool does: it allows the user to connect to an AMF service and make calls, and prints out the results in text and tree forms. So basically, it's a generic AMF client. Since AMF has no service discovery methods, the user needs to define what services are available on the server. When the AMFPHP browser, that comes with a default install, is present on the server, Pinta can use it do discover the available services for you.

In the future, the plan is to build unit testing support into Pinta, so that with one click you can see if your AMF service still responds as it should.

More info about the tool can be found on the Google Code page, http://code.google.com/p/pinta. I hope the tool is useful to some, feel free to comment/request features/report issues.

Tweet This Post

Fortunately, this is possible with ActionScript 3, using the FileStream class's readObject() and writeObject() methods. Here's the code I used to read and write the profiles:

private function loadProfiles():void
{
	var prefsFile:File = File.applicationStorageDirectory.resolvePath(fileName);
	var fs:FileStream = new FileStream();
	if( !prefsFile.exists )
	{
		profiles = new ArrayCollection();
	} else {
		try {
			fs.open( prefsFile, FileMode.READ );
			profiles = fs.readObject() as ArrayCollection;
			fs.close();
		} catch( e:Error ) {
			Alert.show( "Error while loading profiles: "+e.message, "Load error");
		}
	}
}

public function saveProfiles():void
{
	var prefsFile:File = File.applicationStorageDirectory.resolvePath(fileName);
	var fs:FileStream = new FileStream();
	try {
		fs.open( prefsFile, FileMode.WRITE );
		fs.writeObject(profiles);
		fs.close();
	} catch( e:Error ) {
		Alert.show("Failed to save profiles: "+e.message, "Save error");
	}
}

There are a few pitfalls however when loading the profiles. First, the player must be able to tell the class of the objects it is loading. For some reason, it cannot do so unless you specify it explicitly:

package nl.aboutcoding.servicebrowser.model
{
	import mx.collections.ArrayCollection;

	[RemoteClass(alias="nl.aboutcoding.servicebrowser.model.Profile")]
	public class Profile
	{

Secondly, the class MUST have a constructor, otherwise the objects are typed as Object and type casting will fail with a message concernin "Type coercion failed". I often skip the constructor on ValueObjects, and it took some time to figure out.

Tweet This Post

Of course, you can compile your project directly with FDT or FB, but the way it handles its output (how it is generated, where it is placed, what to do after compilation) is much less under your control.

I wanted FDT to automatically upload the generated SWF and other resources after compilation, but only if they had been changed. My deployment folder consisted of these resources:

• preloader.swf
• main.swf
• index.php
• style.css

And an images directory which wasn't changed often. The answer to the problem is Ant. There are many tutorials on how to setup Ant building for your project, it's really quite simple (like this one). I'll add my buildfile at the end of the post, it's self-explanatory. The addition that makes the whole thing upload the changed resource (and only those!) to the webserver is:

<target name="upload">
		<exec executable="rsync">
			<arg line="-avP ${outputBaseDir} masteen.6core.net:www/www.aboutcoding.nl/webroot/"/>
		</exec>
</target>

In this case, I'm creating www.aboutcoding.nl, which is on the server masteen.6core.net in the directory www/www.aboutcoding.nl/webroot. rsync is a utility that comes standard with Mac OS X and most Linux flavours. When it doesn't, it's easily installed.

That's not all though. For easy synchronisation, you don't want to have to enter your password for every upload. In fact, FDT/Ant doesn't handle it well at all if you need to give input during the build process.

Since rsync uses SSH, the solution is to uses public key authentication. You upload your public ssh key (found in ~/.ssh/id_rsa.pub, if you don't have one generate it with ssh-keygen -t dsa) to the server in question, and add it to the file ~/.ssh/authorized_keys there. The server has to support public key authentication though:

RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile      %h/.ssh/authorized_keys

These options need to be in the config of the ssh daemon (usually /etc/ssh/sshd_config).

Then the last thing: your private SSH key is only unlocked with your password. On Mac OS X 10.5 ssh-agent runs in the background, keeping your key unlocked, but on Linux you'll probably have to set it up to do so.

Now you're good to go, automatic uploading with FDT and Ant

Tweet This Post

The result can be found at www.aboutcoding.nl. I embedded the following techniques I wanted to learn:

• Dynamic content through XML
• Indexable by search engines
• Deeplinking enabled
• Embedded fonts with some visual effects
• Full-window flash with graceful resizing

I hope you enjoy it, I sure did creating it

Tweet This Post

• Introduction
• Benchmark setup
• Results
• Discussion
• Conclusion

Introduction

This post discusses the performance of Linux context switches under the Xen hypervisor. Presence of the Xen hypervisor has an impact on the context switching performance, as is shown by benchmarks. I was interested in these benchmarks since I had the feeling a Xen-enabled machine is in general running slower than a non-Xen machine.

So I decided to benchmark a server with and without Xen. I have a server running which isn't in production yet, so it's safe to experiment with. It is an IBM eServer xSeries 345 which I have blogged about before. The relevant specs:

CPU 2 x Intel Xeon 2.4GHz with 512K cache
Memory 4 GB PC2100 DDR RAM, ECC & registered
Disks 2x18G U320 SCSI, 2x36G U320 SCSI, 2x300G U320 SCSI

Benchmark setup

I ran the lmbench benchmark suite, which is in the Ubuntu package repository. The test cases:

• clean: 4 runs with Ubuntu linux 2.6.24-23-server, no Xen at all
• dom0-nodomU: 4 runs with Xen 3.2 and Ubuntu linux 2.6.24-23-xen, from dom0, no domUs running
• dom0-3domU:4 runs with Xen 3.2 and Ubuntu linux 2.6.24-23-xen, from dom0, 3 domUs running
• domU-3domU4 runs with Xen 3.2 and Ubuntu linux 2.6.24-23-xen, from domU, 3 domUs running

The domUs and dom0 were idle except for the benchmark. That is, regular cron jobs were running, but the systems weren't in use for any services. Each test took about 40 minutes, so the tests together are spread out over quite some time mitigating the effects of cron jobs.

Results

The results are too large to include in the post, but they are located here:
xen-benchmark-summary.txt

Below are some of the results where the Xen systems showed a significant difference from the Xen-less system:

Processor, Processes - times in microseconds - smaller is better
------------------------------------------------------------------------------
Host                 OS  Mhz null null      open slct sig  sig  fork exec sh
                             call  I/O stat clos TCP  inst hndl proc proc proc
--------- ------------- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----
clean     Linux 2.6.24- 2400 0.49 0.70 2.97 4.82 5.24 1.15 2.78 280. 1006 2500
clean     Linux 2.6.24- 2400 0.49 0.69 2.98 4.89 5.37 1.14 2.84 279. 990. 2435
clean     Linux 2.6.24- 2400 0.49 0.68 2.97 4.77 5.13 1.15 2.80 280. 1011 2428
clean     Linux 2.6.24- 2400 0.49 0.68 2.96 4.74 5.44 1.15 2.85 280. 1013 2431
dom0-3dom Linux 2.6.24- 2400 0.51 0.70 2.93 4.50 5.16 1.16 2.77 1000 2937 7970
dom0-3dom Linux 2.6.24- 2400 0.50 0.69 2.92 4.80 5.23 1.15 2.70 993. 2902 6591
dom0-3dom Linux 2.6.24- 2400 0.52 0.68 2.94 4.78 5.13 1.16 2.71 972. 2890 6424
dom0-3dom Linux 2.6.24- 2400 0.49 0.69 2.92 4.94 5.09 1.17 2.74 959. 2908 6561
dom0-nodo Linux 2.6.24- 2400 0.49 0.69 2.92 4.55 5.14 1.16 2.70 969. 2786 6283
dom0-nodo Linux 2.6.24- 2400 0.49 0.69 2.91 4.54 5.13 1.15 2.74 961. 2806 6375
dom0-nodo Linux 2.6.24- 2400 0.50 0.68 2.92 4.77 5.14 1.15 2.75 985. 2835 6404
dom0-nodu Linux 2.6.24- 2400 0.50 0.68 2.92 4.77 5.16 1.15 2.79 970. 2779 6360
domU-3dom Linux 2.6.24- 2400 0.50 0.65 1.92 3.26 5.26 0.98 2.24 567. 1757 4108
domU-3dom Linux 2.6.24- 2400 0.52 0.67 1.95 3.26 5.22 0.98 2.29 578. 1742 4072
domU-3dom Linux 2.6.24- 2400 0.49 0.66 2.00 3.30 5.04 0.98 2.17 561. 1767 4127
domU-3dom Linux 2.6.24- 2400 0.51 0.66 1.93 3.27 5.21 0.98 2.25 634. 1880 4361

Context switching - times in microseconds - smaller is better
-------------------------------------------------------------------------
Host                 OS  2p/0K 2p/16K 2p/64K 8p/16K 8p/64K 16p/16K 16p/64K
                         ctxsw  ctxsw  ctxsw ctxsw  ctxsw   ctxsw   ctxsw
--------- ------------- ------ ------ ------ ------ ------ ------- -------
clean     Linux 2.6.24- 3.4500 3.7800 3.9700 5.2000   27.3 9.53000    38.0
clean     Linux 2.6.24- 3.5500 3.7600 4.0000 4.8000   30.9 9.19000    35.6
clean     Linux 2.6.24- 3.5000 3.8200 3.8200 5.0400   30.0 9.98000    38.6
clean     Linux 2.6.24- 3.6000 3.7600 4.1200 5.2100   28.3 9.40000    38.4
dom0-3dom Linux 2.6.24- 7.4700 7.6800 6.7000   10.8   35.1    18.4    38.9
dom0-3dom Linux 2.6.24- 7.6800 7.6200 8.9900   11.1   37.9    18.4    45.1
dom0-3dom Linux 2.6.24- 7.4800 7.6700 8.6300   10.6   38.9    18.2    45.8
dom0-3dom Linux 2.6.24- 7.5400 7.5800 7.5100   10.2   39.4    18.6    47.0
dom0-nodo Linux 2.6.24- 7.6000 7.6700 8.0000 9.3400   37.2    18.1    44.5
dom0-nodo Linux 2.6.24- 7.5600 7.6500 7.7400   11.0   37.1    18.1    44.3
dom0-nodo Linux 2.6.24- 7.4200 7.6900 8.6500   10.5   37.1    17.3    44.7
dom0-nodu Linux 2.6.24- 7.5800 7.7200   10.7   12.7   36.6    17.9    45.1
domU-3dom Linux 2.6.24- 7.1400 7.2800 8.2400 8.1900   35.3    16.8    43.6
domU-3dom Linux 2.6.24- 6.8900 7.2200 7.5500 9.3200   35.5    16.8    43.7
domU-3dom Linux 2.6.24- 7.1500 7.1300 8.3000 9.6700   36.8    16.9    43.7
domU-3dom Linux 2.6.24- 6.9300 7.0700 7.6500 9.1300   35.2    17.2    43.7

File & VM system latencies in microseconds - smaller is better
-------------------------------------------------------------------------------
Host                 OS   0K File      10K File     Mmap    Prot   Page   100fd
                        Create Delete Create Delete Latency Fault  Fault  selct
--------- ------------- ------ ------ ------ ------ ------- ----- ------- -----
clean     Linux 2.6.24-   23.1   18.8   73.6   38.8  1490.0 1.265 3.22970 3.763
clean     Linux 2.6.24-   23.0   18.9   73.7   38.7  1488.0 1.171 3.29740 3.573
clean     Linux 2.6.24-   22.8   19.0   73.9   38.9  1484.0 1.113 3.25150 3.430
clean     Linux 2.6.24-   23.7   19.0   74.7   39.0  1497.0 1.086 3.23500 3.431
dom0-3dom Linux 2.6.24-   24.5   21.1   82.8   41.8  2900.0 1.401 6.54080 3.349
dom0-3dom Linux 2.6.24-   24.7   20.2   76.6   41.9  3000.0 1.533 6.64740 3.360
dom0-3dom Linux 2.6.24-   24.6   20.3   80.1   41.5  2971.0 1.603 6.60690 3.414
dom0-3dom Linux 2.6.24-   25.2   20.5   81.0   42.0  2924.0 1.383 6.65390 3.257
dom0-nodo Linux 2.6.24-   24.0   19.7   88.3   40.9  4551.0 1.421    10.6 3.330
dom0-nodo Linux 2.6.24-   25.0   20.2   87.4   41.6  4572.0 1.489    10.8 3.436
dom0-nodo Linux 2.6.24-   24.5   20.3   88.8   41.2  4627.0 1.539    10.7 3.454
dom0-nodu Linux 2.6.24-   25.4   20.3   88.1   41.6  4636.0 1.554    10.7 3.324
domU-3dom Linux 2.6.24-   20.3   12.9   61.8   25.0  2546.0 1.409 6.08420 3.310
domU-3dom Linux 2.6.24-   20.1   12.8   59.6   24.7  2592.0 1.376 6.06000 3.330
domU-3dom Linux 2.6.24-   19.7   12.9   60.8   24.7  2408.0 1.397 5.84500 3.295
domU-3dom Linux 2.6.24-   19.9   12.9   60.1   24.7  2493.0 1.311 5.89980 3.313

Discussion of the results

As you can see, many measurements are the same for Xen and non-Xen machines alike, like basic integer operations and memory latency and bandwidth. However, a few categories stick out:

• Process creation (fork, exec, sh)
• Context switching
• Local communications latency
• Virtual memory (mmap, page faults)

Process creation
lmbench mesasures process creation latency in three ways: fork, exec and sh. Fork is the simplest way for a UNIX system to create a process: it duplicates the currently running image. exec is a system call that loads a different binary image, and sh calls the shell to execute a binary. In the results we see that process creation of all three kinds is slower by factor 2.8 for systems running Xen. It doesn't matter if they run domUs or not. Strangely, measurements taken from a domU are still slower than no Xen but faster than a dom0.

Process creation implies context switching, which is slower on Xen machines. However, context switching is done in terms of 10 microseconds, while fork takes 1000 microseconds, and only 2 context switches are performed during the fork/exec test, so the context switching overhead is insignificant.

Local communication
Processes on a UNIX system can communicate with each other through pipes. On both ends of the "pipe" a process is listening, and data travels through the pipe. lmbench tests both the bandwidth and the latency of the pipes.

Bandwidth is the same for Xen and non-Xen systems. However, the latency is considerably higher with pipes on Xen systems: 2 to 3 times.

Pipe communication implies context switches, and the context switches needed are significant to the total latency.

Context switching
A multi-tasking operating system (all operating systems today) has many processes running at the same time. CPU time is divided among those processes. When the OS takes process A away from the CPU and puts process B on it, it must save that state that process A is in for when it will resume, and restore the saved state for B. This saved state is called context, and context switching is the task of switching these contexts when a different process gets CPU time. The context is made up of at least CPU registers. A normal system will perform context switches many times per second: my web/mail server averages 400 times per second. Speed of context switches therefore is important.

In the results we see that all Xen systems have twice the latency the non-Xen system has in context switching.

Virtual Memory
Virtual Memory is the extension of memory to auxiliary storage (the swap space). To processes, it seems as if there is more memory present that there really is. Since this auxiliary storage (typically a harddisk) is much slower than real memory, the operating system keeps usage of this storage to a minimum. Memory is divided up into pages, and less-used pages are saved on the auxiliary storage and swapped back to memory when needed.

The mmap function maps I/O onto memory. When reading from this memory, the data is read from the I/O. Since the OS uses its memory so efficiently, data is cached in memory. lmbench uses this to test the caching speed: it only reads data using mmap that was read before (so it doesn't come from the real I/O device). When data is not present, a page fault arises and the data is read from the I/O device.

In the results we see that the Xen dom0 systems have about twice the mmap latency, and about twice the page fault latency. To my knowledge, mmap and page fault do not depend on context switching, but I'm no OS guru. Again we see that the domU system is doing better than the dom0 systems, but still worse than a no-Xen system.

Some more quirks:

• Bandwidth on local communication (pipes) was constantly higher once there were domU machines active
• File creation and deletion was slightly slower with Xen active
• With Xen running but no domUs, mmap was considerably slower than with domUs
• With Xen enabled but no domUs, File Reread was considerably faster than with all other configurations
• Many latencies were lower for domU systems than for dom0 systems

Conclusions

Xen systems are slower at:

• Process creation
• Context switching (and therefore pipes)
• mmap calls

I will use my web/mail/dns server, which is a pretty typical linux server, to find out what these higher latencies mean in real life. I use munin to monitor this server, and it logs forks, context switches, and many other useful parameters.

Forks
The Xen-less system has 4 measurements for forks, the average being 280 microseconds per fork. The Xen systems with and without domUs average 976 microsonds, the domU system averages 585 microseconds.

Context switches
The Xen-less system does a context switch in 3.5 microseconds, so this takes 1568 microseconds per second. The Xen systems took 7.37 microseconds on average for a context switch.

mmap calls
I don't have statistics for the number of mmap calls on my server. However, the Xen dom0 systems take about twice the time the Xen-less system does, and the domU systems add 1000 microseconds to the Xen-less latency. I imagine this would cost you for large amounts of I/O.

Wrapping up
Running Xen impacts performance of some of the basic system calls, like fork(), exec() and mmap(). It also increases the overhead on context switches. Creating a process will take about twice the time with Xen. Process creation latency impacts the performance of networked daemons, since they tend to fork() for every new connection. Every process needs to be created however, so doubling the latency has an impact on general performance.

Context switchces happen on the webserver this website is hosted on about 450 times per second (measured average over last week). A doubled latency over context switches is also a pretty heavy performance impact.

Pipes also have higher latency, but I think this can be blamed on the context switches. Pipe performance therefore will depend on how often a context switch needs to take place between sending and receiving ends of the pipe, but the pipe bandwidth measurements were the same for Xen and non-Xen systems.

mmap also has twice the latency with Xen. I don't know the significance of this, since I'm not a ware of when mmap() is used on my system and when data is simply read() from file pointers. Maybe someone can comment on this

Tweet This Post

To get to the tethering, one must go through these steps:

• Register your iPhone's UUID with apple
• Get the OS 3.0 beta firmware image
• Install the iPhone SDK or at least the USB System Components package that comes with it
• Change the carrier information on you iPhone by uploading a modified IPCC file

I'm registered as an Apple developer since the company where I work is starting to build iPhone apps. So, getting the device registered and downloading the OS 3.0 beta image and SDK was easy.

Upgrading to the new image is done by clicking the "Check for Update" button in iTunes while the Option key is pressed. This allows you to select a firmware image to update to. By selecting the 3.0 image, the device is upgraded. Then the SDK story. I already have the iPhone 2.2 SDK, since I've started writing iPhone apps. For tethering to work however, you need the 3.0 SDK or a part of it. I first installed the pkg that only contains the necessary drivers for USB tethering (the whole SDK is a 2.15 GB download). This file can be found here. Later, when tethering didn't work, I installed the whole SDK, and when it still didn't work, I installed the small PKG again. It finally worked

Then, you need an IPCC file. For T-Mobile NL, you can find such a file in this excellent post. It didn't work for me though, I downloaded this file (thanks to Wiebel).

At first, iTunes and my iPhone would crash when I plugged the iPhone in with tethering enabled. After re-installing the PKG with drivers again, this problem was fixed. Tethering via bluetooth worked after using Wiebel's IPCC file. So, now it works both via bluetooth and USB!

Tweet This Post

To measure how many users a system can serve, I used the following steps:

• Set requirements on response times (like 500ms for 90% of the requests, 1 sec for 98% of the request etc)
• Run the tool with these requirements for a low number of users
• Increase the number of users until the requirements aren't met anymore

Then you can make changes to the system and re-run the tool to see if it still serves the same amount of users, or perhaps more.

The project is at http://code.google.com/p/visim

Tweet This Post

Anyway, the tool has been updated to detect weak BIND 8 and 9 resolvers, so go and check it out!

Tweet This Post

It comes down to this: bind versions up to 9.4.1 use a fixed source port for queries. This source port is determined at startup. Furthermore, they employ a bad PRNG for query IDs. This leads to predictability of the next query ID. One can exploit these two vulnerabilities together to send a resolver false DNS information, since all of the packet can be constructed and sent before a real server could: cache poisoning.

I found this whole thing very interesting, and decided that it would be nice to write a tool that could detect if a certain resolver was vulnerable to this vulnerability or not. In order to be vulnerable, a server would have a) the same source port for multiple requests and b) have predictable query IDs. To detect this, I followed the path Amit Klein described in his path: CNAME chaining. The tool acts as a DNS server. When a resolver sends a request for an A or MX record, it answers with a CNAME record within the domain, triggering another query from the resolver. And then, it does that again and again, up to 10 times. In these 10 times, the tool can determine if the source port was the same all the time, and if any of the IDs was predictable. Only when a query ID is even can the tool predict a range of 10 possible next query IDs (again, Amit Klein's approach).

The tool accepts MX requests. This is because most resolvers do not accept queries from outside their own network. To make them query your domain, one of the possible tricks is interact with the website of your target to find and make it send you email. To send you email, the resolver will perform an MX lookup, exposing it's resolver IP to you, and creating a possibility for fingerprinting. If the resolver is open to the world, you can use a tool like fpdns to fingerprint it further.

I used this project to learn C, so don't expect too much of the programming. The source is here, as a Google Code project. Of course, it's intended for legal penetration testing and research only!

A sample of the output:

chris$ sudo ./nschaind -c lab2.6core.net -f masteen.6core.net -m mail
Chaining daemon listening on 0.0.0.0:53
------------------------------
A mail.lab2.6core.net.	?	192.168.1.2:45682,18216 chaining (1)
A chain00.lab2.6core.net.	?	192.168.1.2:45682,25421 chaining (2)
A chain01.lab2.6core.net.	?	192.168.1.2:45682,33129 chaining (3)
A chain02.lab2.6core.net.	?	192.168.1.2:45682,3140 chaining (4)
A chain03.lab2.6core.net.	?	192.168.1.2:45682,34338 (p) chaining (5)
A chain04.lab2.6core.net.	?	192.168.1.2:45682,41383 chaining (6)
A chain05.lab2.6core.net.	?	192.168.1.2:45682,63194 chaining (7)
A chain06.lab2.6core.net.	?	192.168.1.2:45682,31597 (p) chaining (8)
A chain07.lab2.6core.net.	?	192.168.1.2:45682,64114 chaining (9)
A chain08.lab2.6core.net.	?	192.168.1.2:45682,64825 (p) chaining (10)
A chain09.lab2.6core.net.	?	192.168.1.2:45682,51181 answering (10)
Target 192.168.1.2 correct prediction count: 3, different source ports: 1
TARGET IS VULNERABLE

Tweet This Post

• Part 1: Overview and OS
• Part 2: Disk setup (RAID, LVM)
• Part 3: Xen and domU setup
• Part 4: Networking setup
• Part 5: Intrusion Detection (IDS) setup (coming soon)

Networking and Xen

As for networking, this is a bit different from the normal situation. My colo provider (Coloclue, great joint) is located in multiple datacenters in Amsterdam. They have an IP range for every datacenter, the machines in datacenter 1 will get IPs from a different range than those in datacenter 2. However, if one needs more than one IP address, as I do for my virtual machines, they will assign IP addresses from another range that is not datacenter specific. So, my dom0 has an IP address specific to the datacenter from range A, and the domUs will have IPs from range B. Furthermore, these IPs from range B are routed as /32s, so there is no loss due to network and broadcast addresses.

For dom0, there is no problem. I just configured eth0 with an IP address, the appropriate netmask for the datacenter range, and the default gateway. For domUs however, the story is different. Bridged networking, which is default in Xen, doesn't work now. We'll have to switch to routed mode. In bridged mode, Xen creates a bridge device on dom0, and a number of ethernet devices on top of that. The bridge forwards packets to the ethernet devices at layer 2. With routing however, packets are forwarded on IP level, layer 3. Dom0 will act as a router for the domUs.

First, change the networking mode Xen uses in /etc/xen/xend-config.sxp:

#(network-script network-bridge)
(network-script network-route)

#(vif-script vif-bridge)
(vif-script vif-route)

Now, in the config file for the core domU, core.cfg:

vif  = [ 'ip=w.x.y.z,vifname=core-vif' ]

That's pretty straightforward. I had some problems with the file /etc/xen/scripts/network-route, xend gave a syntax error when restarting. The variable netdev wasn't properly initialized. It seems I'm the only one running in to this problem, must've done something wrong. To fix it, I manually set netdev = "eth0". To make sure that the script has done its work, run this command:

cat /proc/net/ipv4/ip_forward

It should show "1", meaning that dom0 has enabled packet forwarding. Check your iptables setup for restrictions on forwarding, you don't want to become a router for everyone.

Now to configure the domUs. We'll refer to the dom0 IP as dom.0.ip.addr, and domU's IP as dom.U.ip.addr. The file /etc/network/interfaces in domU should look like this:

auto eth0
iface eth0 inet static
address dom.U.ip.addr
netmask 255.255.255.255
broadcast dom.U.ip.addr

pointopoint dom.0.ip.addr
gateway dom.0.ip.addr

And that's it. No strange hacks or obscure configs, but you do need to find out that you need to add the 'pointopoint' directive in /etc/network/interfaces... thanks to tony for finding out!

Tweet This Post

• Part 1: Overview and OS
• Part 2: Disk setup (RAID, LVM)
• Part 3: Xen and domU setup
• Part 4: Networking setup
• Part 5: Intrusion Detection (IDS) setup (coming soon)

Xen and domU's

Setting up Xen on Ubuntu Server is pretty straightforward:

aptitude install ubuntu-xen-server

This meta-package will install all you need. After a reboot, you should be running Xen. Then, for the domU machines. They can be installed using xen-create-image, but I chose the manual setup since I had too much customization to do (in fact, I was too lazy to find out how to do this with xen-create-image). So:

mount /dev/vg0/core-root /mnt
debootstrap hardy /mnt http://nl.archive.ubuntu.com/ubuntu

This installes Ubuntu Hardy into the domU. After this initial setup, it is important to edit the following files to reflect your network settings:

/mnt/etc/hostname
/mnt/etc/network/interfaces
/mnt/etc/resolv.conf
/mnt/etc/apt/sources.list

And make the modules for the current kernel available to the domU:

cp -R /lib/modules/`uname -r` /mnt/lib/modules/

Now we can create a xen config file. Mine looks like this:

kernel      = '/boot/vmlinuz-2.6.24-23-xen'
ramdisk     = '/boot/initrd.img-2.6.24-23-xen'
memory      = '1024'
root        = '/dev/sda1 ro'
disk        = [
                  'phy:vg0/core-root,sda1,w',
                  'phy:vg0/core-swap,sda2,w',
              ]
name        = 'core'
vif         = [ 'ip=94.142.241.53,vifname=vif-core' ]
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'

We should be ready to start the domU now. After booting it, don't forget the following steps:

1. Set a root password
2. Install ssh and libc6-xen
3. Disable root logins via ssh
4. Create a user and put him in the admin group
5. Grant the admin group sudo privileges
6. Setup some firewall rules
7. Move /lib/tls to /lib/tls.disabled
8. Enable swap: swapon /dev/sda2
9. Create a /etc/fstab file containing proc, root and swap

Serial console

There is one thing left: the serial console. Many server administrators will want to use a serial console in case things go wrong with the network. I do, anyway For those who aren't familiar with the concept: a serial console is a login console on the serial port. Typically, you will have a console server in the rack with many serial ports, each one connected to a server. You can then login to the console server via TCP/IP and access the serial consoles of the servers. This way, you can manage the server when its network isn't functioning for some reason (for example, if you're blocked out by an overly eager IDS). Furthermore, the serial console functions as an attached keyboard/VGA: you can access the BIOS and the grub menu.

The serial console must be enabled in a few stages of the boot process in order to be fully functional:

• BIOS
• Grub
• Xen
• Linux

First, the BIOS. How serial console is enabled in the BIOS differs per BIOS, but yours should have an option for serial console somewhere. Then Grub. In order to tell Grub to send output to the serial console and accept input from it, add the following lines to /boot/grub/menu.lst:

serial --unit=0 --speed=9600 --word=8 --parity=no --stop=1
terminal --timeout=5 serial console

The serial defines a serial console at 9600 baud, no parity, 1 stopbit and a wordsize of 8. The parameters may be different for your serial console, make sure you have the same settings on both sides of the wire. 9600 is a pretty low speed, just to be safe.
The terminal command instructs grub to use the 'serial' and 'console' interfaces for the menu. It will wait for 5 seconds for a keypress on either one, and than use that one.

Now, Xen must also send its output to the serial console. Here's a chunk from my menu.lst for the Xen boot:

title           Xen 3.2 / Ubuntu 8.04.2, kernel 2.6.24-23-xen
root            (hd0,0)
kernel          /boot/xen-3.2.gz com1=9600,8n1 console=com1,vga
module          /boot/vmlinuz-2.6.24-23-xen root=/dev/md0 ro console=tty0 console=xvc0
module          /boot/initrd.img-2.6.24-23-xen

The option com1=9600,8n1 again initializes a serial console at 9600 baud, no parity, 1 stop bit. Then, console=com1,vga sends the output to both the VGA and serial interfaces. On the module line, Linux is instructed to use both its normal tty and the serial device xvc0 for output. The device xvc0 is created by Xen. I'm running Xen 3.2 and a xennified Linux 2.6.24, for some older versions of Xen and Linux the serial device is the usual linux serial device, ttyS0. For some even newer versions, the device is called hvc0. To find out which device you should use, boot the system without serial console and check which of the devices ttyS0, xvc0 or hvc0 exists in /dev.

Now, you'll have output of all the stages of the boot process on your serial console. One last thing: you'll probably want to be able to also login on the console. This requires that Linux spawns a getty process (the login process) attached to the serial interface, just as it does for the regular tty's. Under most Linux distro's, this is done by adding the following line to /etc/inittab:

T0:12345:respawn:/sbin/getty -L xvc0 9600 vt102

The init process will then start a getty process attached to /dev/xvc0. For Ubuntu 8.04 and newer, this has been replaced by upstart. The configuration for upstart resides in /etc/event.d, where you can see a config file for every console and the boot stages. Copy the file for tty1 to xvc0 (or, again, hvc0 or ttyS1) and replace tty1 with the appropriate serial device. Make sure to add the -L 9600 option again. The file now looks lik this:

# xvc0 - getty
#
# This service maintains a getty on xvc0 from the point the system is
# started until it is shut down again.

start on stopped rc2
start on stopped rc3
start on stopped rc4
start on stopped rc5

stop on runlevel 0
stop on runlevel 1
stop on runlevel 6

respawn
exec /sbin/getty -L 9600 xvc0 vt100

And that's it, your serial console should be set to go!

Tweet This Post

• Part 1: Overview and OS
• Part 2: Disk setup (RAID, LVM)
• Part 3: Xen and domU setup
• Part 4: Networking setup
• Part 5: Intrusion Detection (IDS) setup (coming soon)

Disk allocation with RAID and LVM

The system has 6 disks: 2 x 18G, 2 x 36G and 2 x 300G. It has a SCSI controller that supports RAID 1 only. However, I choose to use Linux software RAID instead. Using software RAID for RAID 1 doesn't really cause a performance penalty, since the data only has to be sent to 2 disks, no parities have to be calculated like with RAID 5. I want the disks to be fully redundant, so here's the setup. Click the image to see it in full size.
I've created three RAID-1 partitions from the 6 disks, grouping them in pairs. The 18G pair is for the dom0, since it won't have much to do this will be enough space. Then, I want all the remaining space to be available to the virtual machines. And I want to be able to re-allocate space without going through too much trouble. The way to do this is by using LVM: Logical Volume Manager. What LVM does is roughly the following: you assign normal disk partitions which you've given the LVM type ("physical volume, PV") into so-called "volume groups" (VG's). From these VG's you create logical volumes: LVs. These logical volumes can be used as normal partitions again. So, I've created a PV out of both RAID 1 partitions on the 36G and 300G sets. These PV's are joined in one VG. In this VG I've created LV's for disk and swap of all the domU machines.

During setup, I created md0 and md1, the root and swap for dom0. md2 and md3 were created with these commands:

mdadm --create /dev/md2 --level 1 --raid-devices=2 /dev/sdc1 /dev/sdd1

Then, to create PV's, use pvcreate:

pvcreate /dev/md2

The same for md3, and now the md's are ready to be put into a volume group:

vgcreate vg0 /dev/md2
vgextend vg0 /dev/md3

After this, the command vgdisplay gives roughly this output:

 --- Volume group ---
  VG Name               vg0
  System ID
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  8
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               313.28 GB
  PE Size               4.00 MB
  Total PE              80200
  Alloc PE / Size       78592 / 307.00 GB
  Free  PE / Size       1608 / 6.28 GB
  VG UUID               4r52aD-uRod-Bltc-PGMl-2o5g-ubeH-cIRHik

Next, to create the logical volumes with
lvcreate:

lvcreate -L 40G -n core-root vg0

This creates a 40G volume for the root filesystem of the core services domU. It is initialized with
mkfs.ext3:

mkfs.ext3 /dev/vg0/core-root

And with mkswap for the swap partitions. Now, the system is ready to receive some domU's!

Flexibility of LVM

Say I want to replace the 36G drives with bigger ones, which probably will happen some day. If I have enough free space on the 300G's, this can be done without having to move the data to another system. The free space must not be allocated to any LV! You can tell LVM to move the data off a particular PV, then remove the PV from the VG, then create a PV on the new disk and add it to the VG. Also, use the partitions in /dev/mapper instead of /dev/vg0! Using those in /dev/vg0 caused my SCSI host to crash repeatedly.

In my case, I have all space in the VG allocated to LV's. So, first I'll shrink an LV (after halting the domU):

e2fsck -f /dev/mapper/vg0-masteen--root
resize2fs /dev/mapper/vg0-masteen--root
lvresize -L 201G vg0/masteen-root

Now vgdisplay shows the free space:

  --- Volume group ---
  VG Name               vg0
  System ID
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  9
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                6
  Open LV               6
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               313.28 GB
  PE Size               4.00 MB
  Total PE              80200
  Alloc PE / Size       68608 / 268.00 GB
  Free  PE / Size       11592 / 45.28 GB
  VG UUID               4r52aD-uRod-Bltc-PGMl-2o5g-ubeH-cIRHik

So we can remove the old disk (/dev/md2) from the VG:

pvmove /dev/md2
vgreduce vg0 /dev/md2

And the disk can be removed. Likewise for adding a disk:

pvcreate /dev/md2
vgextend vg0 /dev/md2
lvextend -L 230G vg0/masteen-root

And the disk's back in!

Tweet This Post

• Part 1: Overview and OS
• Part 2: Disk setup (RAID, LVM)
• Part 3: Xen and domU setup
• Part 4: Networking setup
• Part 5: Intrusion Detection (IDS) setup (coming soon)

Overview and OS

First of all, I decided to use Ubuntu Server 8.04 LTS, since it has long support for security updates. Ubuntu also is more frequent with releases than Debian, which I've been using up till now. With Debian, I always needed lots of packages from the testing distro, making dependencies and upgrades more difficult. Hope it's better with Ubuntu, it feels just like Debian so far. I'm also going to use Xen for virtualization. On my previous server, I had all services and tasks in one root, and it quickly became a mess. Also, due to a leaky version of Mambo somewhere in a forgotten place on the system, the system was compromised once. By creating virtual machines for different tasks, I'm hoping to keep the system maintainable and more secure. The idea is to have three virtual machines:

1. Hosting: websites, some shell accounts, irssi
2. Core services: DNS, SMTP, IMAP, POP3
3. Dev: a VM to test new stuff in without breaking anything

All three servers will have their own Apache and MySQL daemons. For hosting, it's obvious why. For e-mail, I use MySQL to store domains, mailbox info, aliases and DNS records. I want this MySQL not to be bothered with anything else. For dev, it's obvious as well. 

Then, there's the host system, or dom0 in Xen terminology. Dom0 will have as little tasks as possible: the less it does, the less can go wrong. The real work will take place in the domU's.
As for networking, the colo provider where my server is at (Coloclue, www.coloclue.net) has a particular IP setup to maximize the use of IP addresses. Hosts are assigned IPs from different subnets, leading to some difficulties with Xen and bridging. In the networking post we take care of this problem.

Tweet This Post

The problem
One problem with snort however is that it's only passive: it detects intrusion attempts, reports them to syslog, and sits back. This is not what I want, as I'm not always monitoring my syslog. At the least, I want email alerts when something strange happens. Even better, I'd like the IDS to perform soms actions to restrict malicious traffic once it starts. By blocking all packets from someone who is port scanning, for example.

A solution
A solution to the first problem is to install swatch: a tool that watches the syslog, and sends email alerts. Using regular expressions, you define about which log entries you want to be alerted. Nice, but requires a lot of configuration, probably doesn't catch all alerts you want to know about, and still doesn't solve the passivity of the system. There are solutions that both notify and implement counter-measures, turning snort into an Intrusion Prevention System (IPS) instead of just an IDS. Most well-known are SnortSam and Snort_inline.

A better solution
However, three is also OSSEC. OSSEC is a HIDS (Host-based Intrusion Detection System), and as such it does not watch the network traffic, but just the host. This means it checks the integrity of important files, and watches syslog. The good thing is, it knows about snort, and knows what its syslog messages look like. The better thing is, it is easily configured to perform actions, and by default adds firewall rules to drop traffic from IP addresses snort reports to be port scanning. OSSEC is configured by a large number of well-structured XML files, and operates in a distributed way. You can perform a standalone installation, or a server-agent setup. For my Xen machines, the dom0 is the server and the domUs are agents, reporting to the server. This works great, since when the domU machines report malicious traffic, the dom0 will drop all the traffic from the attacker's IP, protecting all domUs and dom0.

And, OSSEC sends email alerts to inform you about what's going on. What severity an alert must have for the admin to be emailed can be configured. One caveat however, some alerts ignore these settings and are emailed anyway (as can be configured in the alert's XML file).

I've attached two patches to the config to make OSSEC aware of denied AXFR requests by BIND (which it would otherwise email about), and to ignore unexpectedly terminated SSH sessions.

ossec_named_rules.patch
ossec_sshd_rules.patch

Tweet This Post

The problem
Installation of the hypervisor and xend went fine, and I created a few domUs with debootstrap. However, I couldn't get them to boot. After some googling I found that the xennified kernels in Lenny are only dom0 kernels, not domU for some reason. So, I installed kernels from Etch (2.6.18), and everything worked. Sort of.

This week I have to install a new server with Xen however, and I wasn't happy with my mixed Lenny/Etch solution. Since Lenny is testing and a lot changes in the package base, I decided to try Xen with Lenny again. I installed Lenny on a test PC, installed the  xen-linux-system metapackage along with the xen-tools, and kernel 2.6.26-xen. I installed a domU machine with debootstrap/lenny, and tried to boot it. It wouldn't give me a console however. It went through the init scripts until the point where you'd expect a login prompt. Weird.

A solution
Some googling again, and it seems that the udev package, which is required, isn't installed by debootstrap.

1. Mount the harddisk image: mount -o loop /var/xen/domains/[domain]/disk.img /mnt/tmp
2. Enter the mounted filesystem: cd /mnt/tmp
3. Download udev and libvolume-id0: aptitude download udev libvolume-id0
4. chroot into the mounted filesystem: chroot /mnt/tmp
5. Install the packages: dpkg -i *.deb
6. Umount: cd /; umount /mnt/tmp

A better solution
Now you have a working lenny image. The other way is much easier (with thanks to Bernard): when creating the image, pass the --role=udev option to xen-create-image.

[Note: as of this writing, Lenny is expected to be released as stable within 2-4 days. So, the xen behaviour could still change]

Tweet This Post

I was wondering how Apple's new backup feature "Time Machine" works. When looking at a time machine partition with the terminal, all the files that haven't changed are simply hard links. However, when I listed them with 'ls -al', I saw a an attribute I haven't seen before:

-rw-r--r--@ 2 chris staff 14M Jan 27 13:58 stoel.mov
-rw-r--r--@ 2 chris staff 12M Aug 9 05:31 tony vs paul.flv
-rw-r--r--@ 1 chris staff 59M Jan 28 20:40 visiting shoes.mov


Also, this '@' sign is present on other files sometimes. The man page of ls says:

-@ Display extended attribute keys and sizes.


I also discovered this attribute on another bunch of files I recently downloaded. The strange thing about these files was that I couldn't modify or move them: the Finder (and 'mv') wouldn't allow me to. After some googling, I found this link that explained why.

The '@' sign indicates that there are extended attributes in place on the file. You can list them with ls -@, and you can get and set them with 'xattr'. On the files I couldn't modify, I saw there was a 'com.apple.quarantine' attribute in effect. After removign it with 'xattr' I could move them again.

Back to Time Machine. The files in Time Machine have many attributes, for example:


Appelflap:Stop Motion chris$ xattr -l stoel.mov
com.apple.FinderInfo:
0000 4D 6F 6F 56 69 53 74 6F 00 00 00 00 00 00 00 00 MooViSto........
0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

com.apple.quarantine: 0000;479afced;Safari;51D09ED4-1818-4136-A801-52237BD2E12E|com.apple.Safari
Appelflap:Stop Motion chris$


While these attributes are binary, they reveal some information. The "com.apple.metadata:kMDItemWhereFroms" attribute shows the URL I downloaded the movie from, which is keepvid.com. The other properties have to do with version control in time machine.

Interesting stuff. I haven't found the time yet to dig deeper into time machine and see what it does with diffs and all, for now it seems like it just hardlinks unchanged files and uses these 'extended properties' for version management. Which explains why it only works on HFS+

Tweet This Post

Have fun!

Tweet This Post

Eventually I succeeded, using maildrop (http://www.courier-mta.org/maildrop/) instead of the postfix virtual delivery agent. Maildrop now invokes spamassassin based on a per-user (virtual user) config file, and drops spam to INBOX.Spambox. Cool!

However, I also wanted some nice statistics of the filtering. It would be nice to see what percentage of the mail coming through the server is spam, and which users get the most spam. I couldn't find a nice solution for this however. The solution that came the closest was sa-stats.pl (http://0wned.it/view/2007/03/17/sastats_in_cvs/). It prints out the statistics I want, based on logfile analysis. However, it doesn't collect statistics over a longer period.

So what I did is this: I modified sa-stats.pl to save the found statistics into a MySQL database. Using MySQLs aggregate functions (SUM, AVG, COUNT) it is very easy to get statistics of spam and ham, and totals. I wrote a PHP program to print these statistics.

So, here is the modified script, and here the SQL file that creates the tables it uses. Don't forget to change the database settings in the pl file around line 148!

And here is a screenshot of what the output looks like.

Enjoy!

Tweet This Post


function changeText( )
{
  var node = document.getElementById('nodeToChange');
  node.innerText = 'Hello, World!';
}


This piece of code is pretty straightforward: the function changeText changes the text within the node with id="nodeToChange" into "Hello, World!".

This code won't work in Firefox. You'll see nothing change in the node. Firefox uses the textContent property to change the contents of a text node. So, one would expect the solution to be to detect the browser that the visitor is using, using some advanced browser detection script or your own routines. The name of the browser is available in the navigator.userAgent property. But most of the time, this is a very poor solution. Let me explain why.

On the surface, it may seem that browser detection works. If the user agent is Firefox, you use the textContent property, otherwise you use innerText. But, who guarantees that Firefox is the only browser using textContent? Maybe there is some obscure browser for mobile devices using the same convention. Or, what if a new version of Firefox that will be released in two months will have switched to innerText for compatibility? Then you'd have to change all the scripts you've put online for your clients.

The better solution here is object detection. Instead of looking at the user agent, you look at what feature is supported. Thus, it doesn't matter anymore which browser is being used. In JavaScript, functions can be treated like objects. The following expression:


document.getElementById;


Would evaluate to true, since there is a member of document called getElementById. It isn't until you put braces and optionally some arguments behind the name that the function is executed. So this;


document.getElementById;

Checks if the function exists, while this:


document.getElementById('test');


Actually executes the function. In the same way, properties can be tested. This javascript feature lies at the heart of object detection. Using object detection, our solution to the innerText versus textContent would be:


function changeText( )
{
  var node = document.getElementById('nodeToChange');
  if( node.innerText )
  {
    node.innerText = 'Hello, World!';
  } else {
    node.textContent = 'Hello, World!';
  }
}


This code will run nicely in all browsers that support one of both properties, current browsers and browsers yet to come.

Tweet This Post

The Saumsung D500 officially doesn't have iSync support. Or the other way around, Apple iSync officially doesn't support the D500. However, after googling for a while, I found out that Nova Media has a set of iSync plugins that adds additional phone support, including for the D500. I purchased the software (€10,-, not too bad) and gave it a try.

However, this didn't seem to be enough. When attempting to synchronize, iSync would give me the following error:

There was an error getting data from the phone. The synchronization may have been cancelled on the phone.
Device "chris" synchronization failed

I tried several things, including resetting the phone to default settings, but nothing really seemed to work. After a while, I e-mailed Nova Media requesting support. They were very quick to respond, and advised me to turn bluetooth on and off, remove some iSync, iCal and Address Book preference files, reset the D500, and try it again. I tried these and many other things, but nothing worked.

However, the solution. After clicking around for a while, I found what the problem was. Or rather, I found how to work around it. The synchronization would work if I selected 'Merge data on handheld device and computer' instead of 'Erase data on device then synchronize' as the first-time synchronization action. Perhaps it's a bug in the Nova Media D500 plugin or something like that, but now synchronization works fine.

I'm using Nova Media iSync phone plugins version 4.0.4, perhaps this will be fixed in future releases.

Happy synching!

Tweet This Post