Backup & Restore HCL Me X1

If you are planning to experiment with any Android phone/tablet, the first thing you would like to do is take a backup of it just in case things get messy. Typically in such situations you would like to take a backup of not only the user-data but also the operating system.

In this post I will be discussing on how to create a backup and restore the backup. You don't need to root your device for this since adb already has root access in HCL Me X1. But you will need fastboot which can be obtained by compiling the Android source code.

Remember that the source code can be built only on a Linux distribution, so make sure you have one (I would recommend Ubuntu 11.10). fastboot will be available at <source_code_dir>/out/host/linux-x86/bin/fastboot after the compilation. If you don't want to compile the source, you can try a compiled version of fastboot from the link specified at the end of the post; I built it on Ubuntu 11.10.

Replacing the Recovery Image

Before replacing the recovery image, create a backup of it in case you want to revert it back. Open the shell of the device using adb shell. Find the mount-point of the recovery image using cat /proc/mtd. There will be an entry like - mtd6: 00a00000 00200000 "recovery"

The recovery image can be created by reading the contents of the recovery mount-point and writing it into a file. This can be done using the following command - cat /dev/mtd/mtd6 > /sdcard/recovery.img

The recovery image is written to the sdcard in this case, you can redirect it anywhere you want to. The recovery image can be flashed back using fastboot. For this you have to boot the device into the bootloader using adb reboot-bootloader. The image can be flashed using the following command - fastboot flash recovery recovery.img

You can backup other mount points like system, userdata, etc. using this method but it's a lot easier to do so using the ClockworkMod's Recovery Image. You can download it from the link specified at the end of the post. Flash it using the same fastboot command as above.

Creating a Backup

Reboot the device into the ClockworkMod's recovery mode using adb reboot recovery. This console has options to backup and restore the operating system. However it does this using the sdcard, so make sure you have one in the device before you proceed.

From the recovery console, select Backup and Restore > Backup. After a series of messages the ROM will be backed up.

Restoring a Backup

From the recovery console, select Backup and Restore > Restore. Select the backup you want to restore. After a series of messages the ROM will be restored.

In case you are looking for the stock ROM you can find it here. It's not the exact stock ROM but an unrooted factory reseted ROM. And the original recovery image, ClockworkMod's recovery image and the compiled version of fastboot is available here.

Rooting & Unrooting HCL Me X1

Android is primarily an operating system based on the Linux kernel. So similar to other Linux based operating systems Android has a special user account known as the root or super-user. Unlike other user accounts which have limited control over the operating system this account has complete access to the system.

However several (almost every) carriers and manufacturers don't allow users to access this account mainly due to security concerns. Few carriers and manufacturers also do this because they don't want users to access features for free when they can sell them :). Rooting is a process that allows users of phones/tablets to gain this privileged control (the root access). Here is an article with the Top 10 Reasons To Root Your Android Phone.

Though rooting your device is not illegal it might void the warranty of the device. So think twice (make that trice) before rooting your device and make sure you have a good reason to do so. Read these articles on The dangers of rooting your Android phone and What is Rooting on Android? The Advantages and Disadvantages

The process of rooting a device changes from device to device. It usually involves using an exploit to either gain temporary root access or to flash a custom recovery image to attain permanent root access. There are popular exploits like GingerBreak, psneuter, etc. and applications like GingerBreak.apk, SuperOneClick, etc. for rooting a device.

If you have read the articles and still want to go ahead let me remind you once again that trying to root your device might brick the device. I am not responsible if you brick your device, so proceed at your own risk.

Rooting the Device

You don't require any exploit or any application to root HCL Me X1 and in-fact vulnerabilities like GingerBreak have already been patched. However it is possible to root the device using the adb shell so make sure you have the Android Debug Bridge (ADB) on your system.

Connect the device through the USB and open the shell using adb shell at the command prompt. If you are seeing the # here then that indicates that your adb shell has root access. Instead if you are seeing a $ this process won't work for your device. Please put in a comment if you see $ so that me and the other readers know that HCL has changed something.

Typically in most of the devices the adb shell does not have root access. However, looks like the Android build of Me X1 isn't a production build. Not sure if this was done knowingly or unknowingly by HCL but either way the tablet has root access through the adb shell. You can confirm this by trying one of these -

• Type adb root at the command prompt, it will show you a message stating that adbd is already running as root
• Type getprop ro.secure at the adb shell, it shows 0
• The ro.secure property can also be examined from the /default.prop. If this is 0 then it indicates that the adb will login as root by default. Note that this file cannot be changed since it's a part of the ramdisk

Though the adb shell has root access, the su command which is used in Linux doesn't work in Android since there is no su binary in Android. Download the su binary and the Superuser application from here (backup).

There are two steps for rooting the device -

1. Copying the su binary to the /system/bin directory

Since this directory is on the Android PATH, applications will be able to call su directly. However since the /system is a read-only file system (type mount at the adb shell to check this), the file cannot be directly copied. Push the su binary from the system/bin directory in the zip to a temporary folder of the device (like /data/local/tmp) using adb push. However note that the su binary does not work directly from this directory since applications on the /data file system do not have the setuid bit on. Remount the /system with read-write capabilities with the mount -o remount -rw -t yaffs2 /dev/block/mtdblock2 /system command (check the actual device mount-point using mount). Copy the su binary to /system/bin and set the permissions of the binary using chmod 4777 su.

2. Install the Superuser.apk from the system/app directory of the zip

Install the application using adb install or transfer the file to the sdcard and install it from there. The Superuser application intercepts the calls made to the su binary and informs the user about the usage so that unauthorized applications cannot attain root access.

Unrooting the Device

To unroot the device, uninstall the Superuser.apk and delete the su binary from the /system/bin directory.

A Primer To Android Development

As most of you know, Android is a mobile operating system initially developed by Android Inc. and presently being developed by the Open Handset Alliance led by Google. Android consists of a modified Linux kernel and software built on the Apache Harmony framework (an open source Java implementation). The best thing I like about the Android platform is its versioning. The code names are released in alphabetic order of dessert items - Cupcake, Donut, Éclair, Froyo, Gingerbread, Ice Cream Sandwich and Jelly Bean.

I am a newbie myself when it comes to Android, so the past few days I was trying to understand the various development models of Android. During my search I came across several good tutorials. This blog post is primarily to consolidate these tutorials for beginners like me.

Android, like most open source Operating Systems allows developers to look at development from two different perspectives - working on Android (Application development) and working with Android (Operating System development, not possible with proprietary Operating Systems). Google understands this and therefore has two different sites for Android developers - Android Developers (has the Android SDK to build applications) and Android Open Source Project (has the Android source code).

Android Developers

The Android Developers site provides various Development Kits to create applications for the Android platform. The Android SDK allows developers to write applications for the Android platform in a slightly modified version of Java which runs on a virtual machine called Dalvik.

The Android SDK can be installed by following the instructions specified here - Installing the SDK. And in case you want to start programming right away, here's a classic Hello World tutorial.

The Java APIs are pretty much what most developers use. However there might be situations where you would like to harness the true potential of the underlying Linux kernel through a low level language like C. Android does allow developers to do that but they cannot call the C binary directly as an application, they will need to go through Java with the JNI (Java Native Interface). This methodology of development is supported by the NDK (Native Development Kit).

The Android NDK can be installed by following the instructions specified here - Installing the NDK. The NDK does come with the Hello World example but it isn't as easy as the regular one.

The most useful tool Android provides developers with is the Android Debug Bridge. It is a command line utility that supports several commands like pulling/pushing files to and fro the device, installing applications, etc. This is one tool that every Android developer will have to use irrespective of whether he is developing applications or playing with the source code. Please note that the adb is part of "Platform Tools" which can be installed through the Android SDK Manager. If you stuck trying to install adb, read this.

Android Open Source Project

The Android Open Source Project site provides access to the source code of Android. A majority of the operating system infrastructure of Android is distributed under the Apache License and the core kernel is licensed under the GPLv2 License.

Certain developers use the Android Open Source Project to find vulnerabilities in the Android operating system and exploit them to execute tasks which cannot be done normally like attaining root access. Examples of these exploits are GingerBreak, psneuter, etc. Typically these vulnerabilities are patched for security reasons.

A majority of developers use the Android Open Source Project to modify and customize the operating system to create custom images. Several vendors add device drivers for their hardware and install pre-release applications to the operating system before flashing it onto the device. There have also been situations wherein vendors have customized the UI to suit their product lines before flashing.

You can obtain the complete source of the Android operating system by Downloading the Source Tree. There is an awesome tutorial on building Android 2.3.7 on Ubuntu 11.10 (64 bit) here. Even though the AOSP site specifies instructions on compiling the code, they are a bit outdated so visit the above link if you are stuck anywhere.

Underneath the layers of application frameworks, Android is primarily a Linux kernel and therefore is built on a C library. Unlike the traditional Linux based operating systems, Android uses a custom tiny libc called bionic. C programs can be written in an Linux environment running on x86/x64 architectures and cross compiled to Android running on ARM by linking Bionic to gcc. Here is an excellent tutorial which shows you exactly that. The compiled code can be run using adb shell. However note that codes compiled this way aren't recommended to be used with Java applications, the NDK should be used instead.

HCL Me X1 - A Customers Review

HCL Me X1 is HCL Technologies latest tablet and is one of the cheapest tablets available in India priced at Rs. 10,490. HCL released the X1 on October 14th. This is HCLs third tablet in the Me Series after the AE7-A1 and AM7-A1. I ordered the tablet on 19th through the HCL Store and received it this Tuesday. The entire week I have been toying with the tablet and here are my initial impressions on the Me X1.

This is my first Android tablet (my first Android device in fact) and one of the main reasons I bought the tab was to put up with the Android craze and have my share of fun with the Android platform. So expect to see several posts on the Android platform and the HCL Me X1 on this blog :).

I have been researching Android budget tablets for quite some time and finally decided to get myself the Me X1 (The Lenovo IdeaPad A1 was another option but decided to go with Me X1 after Lenovo priced it at Rs. 15000). Before going deeper into the review let me start by saying that the HCL Me X1 is a good tablet from a price to performance perspective and does a decent job in gaming, music and movies.

The specifications of the HCL Me X1 can be found here. Though HCL hasn't specified the actual Application Processor they are using, it is expected to be the Marvell ARMADA 610.

As evident from the above specifications, the HCL Me X1 doesn't have a SIM card slot and so doesn't support 3G directly. However it does support 3G through a USB dongle. This wasn't much of an issue for me since I was planning to use my Nokia X6 as a WiFi hotspot (haven't tried this yet). The touch responsiveness of the device is good and Gingerbread performs well on the device thanks to the 1 GHz processor.

However the HCL Me X1 does have its shortcomings,

• A major drawback of the Me X1 is that it doesn't have the Android Market on it. Whatever HCL may say, its App Store is no match to the Android Market
• The device works for around 6-8 hours (usage + standby) with a single charge. I wasn't satisfied with this as my Nokia X6 gives me much more than this. However benchmarking a tablet to a phone isn't correct
• The display would have been better if it was a slightly brighter

The biggest weakness of HCL is their delivery service. When most online sites ship within 2-3 days, HCL as a policy takes a minimum of 7 to 10 days to ship the product. However they sent me the devices in 7 days when I pestered them a lot. In fact I would recommend people to buy the device at a local Sangeetha store where you can get it immediately. On the other hand their customer care was good, especially the Live Chat, they answered a lot of questions and followed up on phone as well.

I think the biggest technical improvement opportunities the Me X1 has are the integration of the Android Market and getting out an Ice Cream Sandwich as soon as possible. This should attract more customers towards the product.

Before I wrap up, here are some excellent reviews on the tablet which helped me take a decision on the purchase -

• Arpan Deb's review on the HCL Me X1
• iGyaan's video review on the HCL Me X1
• HCL Me X1 - To Buy or Not To Buy

Update 2011/11/10

After several discussions at arpandeb, we finally concluded that the actual Application Processor being used is the Telechips TCC 88xx. We also found striking similarities between HCL Me X1 and Coby Kyros MID7022.

Named Groups In Regular Expressions - Java 7

One of the new features added in Java 7 was the introduction of named groups in regular expressions through the java.util.regex package. This article covers the various features of named groups in regular expressions and their syntax in Java.

Parts of a Regular Expression can be grouped together by placing them inside round brackets. A major advantage of grouping is that various regex operators can be applied to these groups. Grouping is also useful for back-referencing a match. This allows developers to write regular expressions involving complex repetition patterns more easily.

The value of the captured group can be retrieved directly or be referenced in replacement patterns for changing the format of an input string. Though Java supported Regular Expressions and Grouping through the java.util.regex package from Java 1.4, it was limited to numbered groups.

Here's an example showing how numbered groups can be used in Java. To read more about Grouping and Backreferences, check this out.

Parallel to numbered groups exists named groups. Named groups do not change the underlying concept of grouping and back-referencing but provide more readability to the regular expressions and make back-referencing easier for developers since remembering names is easier than remembering the relative position of a group.

Named groups are already present in languages like Perl, Python and .NET but wasn't available to Java prior to Java 7. Well, better late than never.

The newly added RegEx constructs in Java 7 are as follows -

• (?) defines a named group "name"
• \\k back-references a named group "name"
• ${name} can be used to reference a captured group in a replacement string
• group(String name) returns the value of the captured group

Here's an example showing how named groups can be used in Java 7. To read more about Named Groups, check this out.

However if you are using Java 7, I would strongly recommend using the java.util.regex package but if you are using a Java version older than 7, there are other alternatives available for named groups in Java like Named-RegExp and JRegex. These alternatives were highly recommended by other developers at stack overflow.

DC Comics Reboots The Universe

Today's a big day in the DC Universe. For people wondering what DC Comics is - DC Comics Inc. is a subsidiary of Warner Bros. and is one of the largest and most successful companies operating in the comic books industry. It introduced a majority of the popular superheroes we have grown up with like Superman, Batman, Green Lantern, Catwoman, etc.

Effective this month, their entire comic series has taken a major overhaul and all DC Universe titles being cancelled or restarted at #1. Titles which have been around for more than 70 years like Action Comics and Detective Comics, each having around 900 issues credited to them are also being rebooted making it a very big thing in the comic world.

The DC Comics Reboot (AKA The New 52) has been termed as the biggest thing to happen to comic books since Crisis On Infinite Earths (which happened 25 years ago, before I was born :P). This revamp is going to affect not only the storylines but also the way books are being marketed.

Things are changing a lot in the comic world - print is dying, comic book sales are going down year after year and the revenue streams for comic book writers are going dry. With movies like The Dark Knight, X-Men, Superman, etc. making big impressions at the box office, their sources of inspiration going down would be seem very strange but it's true.

I personally would attribute to two major reasons -

• Complex Story Lines - Movies aren't driving comic sales since movies are far far behind the comic series. Today a newbie can understand The Dark Knight or play Batman - Arkham Asylum but when they grab a copy of Batman comic off the shelf they seem out of sync
• Digital Media - Complex Story Lines are a problem but a far bigger problem is the influence of digital media on the world. People today are far more addicted to sites like Facebook, Twitter, Youtube, etc. and are hooked to devices like iPads, Nooks, Kindles, etc. than reading a physical book. Gone are the good old days when we visited a store to grab a book; we today either download an ebook or wait the movie to come out ;). I don't blame anyone but this is price which we have to pay for embracing technology. The bigger point is that organizations have to understand this and adapt to it

The DC Reboot primarily addresses both these issues; a reboot means that things start from beginning and today's generation can understand the storyline from scratch without having to dig back for older books. On the darker side, this will certainly have a mixed reaction from the existing collector base which has to be dealt carefully.

Also every DC Comics book from here on out will be digitally distributed "day and date". This means that on the same day individual issues hit the shelves of the comic stores, they will also be available to read digitally. The digital distribution of the comic books is something which I liked a lot but I can foresee a major problem of piracy :(. But still it's a great move and if the pricing is maintained properly for people around the globe, it can create a huge impact for the comic world.

For those of you wondering about character origins and histories, it's still unclear what's going to happen - a few of them may be left untouched while the others may see a major change. For instance I didn't see the Martian Manhunter in the Justice League #1.

This move will definitely stir up some hype and I can't wait to see how Marvel responds to this :). Either way, this year's Comic Cons are going to have a lot to talk about and this move will surely have an impact on the future of the comic book industry.

Check out the trailer of DC 52 below -

Check out these cool articles on the DC Reboot -

• DC Comics’ New 52 Reboot - The Complete Guide
• DC Comics hopes revamped heroes and digital will save the day

Remote Debugging Java Applications

After my previous article about Remote Debugging .NET Applications using Visual Studio 2010, I was curious to see how Remote Debugging works for Java Applications. This blog post covers Remote Debugging Java Applications using NetBeans 7.0.

The basic concepts of Remote Debugging are the same for .NET and Java but the process of setting up the host and remote computers varies between Visual Studio and NetBeans. Similar to the .NET application used in the previous post, the screen shots correspond to a simple Java application which would popup a MessageDialog on a button click. Get the code here.

Before configuring the host and remote computers, it is vital to understand the Java Platform Debugger Architecture (JPDA). JPDA provides the infrastructure you need to build end-user debugger applications for the Java Platform. It includes the following APIs broken into three layers -

• Java Debug Interface (JDI), a high-level Java programming language interface including support for remote debugging
• Java Debug Wire Protocol (JDWP), which defines the format of information and requests transferred between the process being debugged and the debugger front end
• JVM Tools Interface (JVM TI), which is a low-level native interface that defines the services a JVM provides for tools such debuggers and profilers

Configuring the Remote Computer

Run the Java application using the -Xdebug and -Xrunjdwp options from the command line.

java -Xdebug -Xrunjdwp:transport=dt_socket,address=6000,server=y -jar RemoteDebugging.jar

Here's a description of the options of the java command -

Option	Description
-Xdebug	Enables debugging support in the VM
-Xrunjdwp	Loads in-process debugging libraries and specifies the kind of connection to be made

The -Xrunjdwp option has several sub options. Here are the descriptions of the ones that are used above -

Option	Description
transport	Name of the transport to user in connecting to debugger application
address	Transport address for the connection If server=n, attempt to attach to debugger application at this address If server=y, listen for a connection at this address
server	If y, listen for a debugger application to attach If n, attach to the debugger application at the specified address

By default the application starts in suspended mode. In suspended mode the application waits for a debugger to attach itself to the server at the specified port before the application starts.

Configuring the Host Computer

The host computer is the system running NetBeans 7.0. Open the code of the application in NetBeans and select "Attach Debugger". Specify the Connector as SocketAttach, the Host as the hostname of the remote system and the Transport and Port as specified above.

Specify the breakpoints in the code and they would hit appropriately. There are two major bottlenecks in Remote Debugging -

• The code from which the executable was built should be available at the time of debugging
• Applications cannot be configured for Remote Debugging at runtime. The -Xdebug option must be specified at the instantiation of the application, making debugging live production code difficult

Before I conclude, here's an article from OTN (Oracle Technology Network) on the Java Platform Debugger Architecture. Visit it if you would like a deeper insight into Java Debugging. The schematic of the JPDA is from a weblog, check it out here.

Remote Debugging .NET Applications

From the past few days I have been stuck in resolving a bug which cropped up in one of my applications after it went to Production. The worst part was that it was a machine-specific issue and we couldn't reproduce it in any of our development systems. While trying to find a solution for this, I came across the concept of Remote Debugging.

Remote debugging is debugging a remotely running application through a development environment running on a system other than the one running the app. This is done by connecting the remote system to the system containing the development environment (in turn the debugger) through Sockets. Theoretically this is achieved in two steps -

• The remote computer would open a socket and listen to debug instructions through it. These instructions are feed to the application that is being debugged and the application responds appropriately
• The debugger would connect to the socket opened by the remote system and send instructions through it

Remote debugging can be used in many live situations. Examples include -

• Debugging a machine specific issue which can't be reproduced on systems running the development environment (my exact requirement :))
• Debugging applications which can run only on the server machine due to dependent third party libraries that cannot be used in the development machine

In this blog post, I will be focusing on Remote Debugging .NET applications through Visual Studio 2010. The screen shots correspond to a simple .NET application which would popup a MessageBox on a button click. Get the code here.

Configuring the Host Computer

The host computer is the computer containing the development environment. This is the system running Visual Studio 2010. To enable Remote Debugging, make sure the following ports are open on the host computer -

Ports	Protocol	Description
135	TCP	Required
500, 4500	UDP	Required if your domain policy requires network communication to be performed through IPSec

IPSec (Internet Protocol Security) is a protocol suite for securing IP communications authenticating and encrypting each IP packet of a communication session. Read more about IPSec here. However I am not going cover IPSec in this post.

You can check if the ports are open by using the telnet command at command prompt (telnet isn't installed by default on Windows 7. Install it from the "Turn Windows features on or off" section in the Control Panel).

Configuring the Remote Computer

To configure the remote system for debugging, either download and install the Microsoft Visual Studio 2010 Remote Debugger or copy the folder Remote Debugger from the Visual Studio install path, which is typically C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE. The Remote Debugger is available for three configurations - x86, x64 and ia64.

Visual Studio takes the .pdb files for debugging from the remote system for managed code. So copy the .pdb from the bin/Debug folder of the project to the remote system. Put this folder on share so that Visual Studio can access it later.

To enable Remote Debugging, make sure the following ports are open on the host computer -

Ports	Protocol	Description
135, 139, 445	TCP	Required
137, 138	UDP	Required
500, 4500	UDP	Required if your domain policy requires network communication to be performed through IPSec
80	TCP	Required for Web Server debugging

That's it; both the host and remote computers are up and ready for Remote Debugging.

Start the application on the remote system if it isn't running already. Also start the Visual Studio Remote Debugging Monitor on the remote system using the msvsmon.exe from the appropriate configuration folder in the Remote Debugger directory (The screenshot uses the x86 version).

Open the code in Visual Studio on the host system and select "Attach to Process". Typically Visual Studio identifies the .pdb file of the remote system if it can. If it doesn't, open the Modules screen from Debug -> Windows -> Modules. Check the Symbol Status of the .exe file which is being debugged and if it shows "Cannot find or open the PDB file" right-click on it and select "Load Symbols From" and specific the share path through the "Symbol Path".

Specify the breakpoints in the code and they would hit appropriately. There are two major bottlenecks in Remote Debugging -

• The code from which the executable was built should be available at the time of debugging
• The above mentioned ports should be open for debugging. Most of the developers wouldn't have administrative privileges on the client systems, so they would require people with sufficient privileges to open up these ports. However several organizations wouldn't prefer opening up ports for incoming connections

Before I conclude, here's an article from MSDN on how to Set Up Remote Debugging. Visit it if you would like a deeper insight into Remote Debugging.

TVPs Vs XML - An Analysis Through .NET

A common requirement of .NET applications that connect to databases like SQL Server is passing Lists and Arrays to the database server. Since most of these databases are relational, transferring these Lists of Objects isn't really straight-forward.

One way of passing these objects would be to send individual properties of these Objects to the database server, one object at a time. Obviously this is highly inefficient. Understanding this, various database management systems have provided various approaches to achieve this.

There are two ways of passing such content to SQL Server -

1. Generating an XML from the List and passing it as a parameter
2. Creating a Data Table from the List and passing it as a Table Valued Parameter

Passing Lists and Arrays as XML to SQL Server has been a common approach since SQL Server 2000. From SQL Server 2008, Microsoft introduced a new way of passing Lists and Arrays - Table Valued Parameters.

In this post, I will be evaluating the performance statistics of XML and TVP though a .NET application. I will be doing this through the RetrieveStatistics method of the SqlConnection class. This method retrieves the statistics of an operation through an IDictionary which contains various performance metrics.

This happens only when the StatisticsEnabled property is set to true before executing the query/stored procedure. These metrics can be reset using the ResetStatistics method of the SqlConnection class. Here's a typical usage of this method -

The dictionary contains several provider statistics. There are 18 values that can be obtained from the Microsoft SQL Server provider. A detailed analysis of the metrics that can be obtained from the dictionary is available here - http://msdn.microsoft.com/en-us/library/7h2ahss8(v=vs.80).aspx.

The .NET application that is being used in this post to compare the performance of TVP and XML captures these 18 metrics into an object. However the final analysis focuses on three of these properties - Execution Time, Bytes Sent and Bytes Received.

The application primarily inserts records into a table. The number of records and the value of the records are decided by the application. To understand the effect of load, the numbers of records are raised from 25 to 2500 in gradual steps of 25. For an accurate measurement each step is repeated for 25 times and an average of the metrics obtained is considered.

Here are the graphs obtained for Execution Time, Bytes Sent and Bytes Received -

Execution Time (Records on X-Axis and Time on Y-Axis)

Bytes Sent (Records on X-Axis and Bytes on Y-Axis)

Note - The Bytes Sent for OpenXML and Nodes are almost the same.

Sent Received (Records on X-Axis and Bytes on Y-Axis)

Note - The Bytes Received for TVP and Nodes are the same.

The observations that I could conclude are as follows -

• The execution time for TVP is less than XML (Using OpenXML took more time than Nodes). The execution time of OpenXML was far higher than TVP
• The bytes sent for XML was higher than for TVP. This is probably due to the XML tags that had to be added for the transfer
• The bytes received in all the three cases were constant irrespective to the bytes sent. Though the number of bytes received for TVP and Nodes was the same, the number of bytes for OpenXML was slightly higher. This might be due to the procedure calls of sp_xml_preparedocument and sp_xml_removedocument

On a whole, TVPs look more promising than XML. However this was a very preliminary test and the actual results might vary in live environments.

The SQL scripts and the Visual Studio project used in this post can be found here and the raw numbers are here.

Compiling Visual Basic, C# & F# Code Into A Single DLL Using msbuild

Recently I had a requirement in which a C# file had to be included in a VB.Net project. I knew I couldn't add the file directly to the project, so I cut over the C# file to VB.Net. At first this seemed pretty obvious since Visual Studio maps every project to a programming language and uses an appropriate compiler to compile the files in the project.

But giving it a deeper thought, I felt something didn't fit the picture. Every project, rather every file that we compile using the .NET Framework generates Common Intermediate Language which is used by the Common Language Runtime. It is because of this CIL that we have language interoperability in .NET. So theoretically speaking, code written in different languages should be able to exist in a single assembly.

Though I couldn't find a way to directly add a C# file into a VB.Net project, I found a way to create a DLL from compiled VB.Net and C# code through a concept of the .NET framework called Multi Module Assemblies.

In .NET, the minimum unit of deployment is an assembly; Dynamic Link Libraries (DLLs), Executables (.exe), etc. are all assemblies. An assembly can contain multiple files like resource files, the manifest, etc. It can contain files of another type called netmodules. A netmodule is a unit of compilation. It cannot be deployed directly but can be linked into an assembly. netmodules written in different .NET languages can be linked into a single assembly.

Unfortunately Visual Studio wasn't built for Multi Module Assemblies, hence there's no way to create a netmodule from a project in Visual Studio. However msbuild allows us to create a netmodule from the projects. There are typically three changes that have to be done to the .proj files (.fsproj, .vbproj, .csproj, etc.) for msbuild to generate a netmodule -

1. Changing the OutputType to Module. Visual Studio doesn't recognize this output type, so even when you change this in the .proj files, Visual Studio won't be able to show it in the Project Properties
2. Exclude the AssemblyInfo file from the project. netmodules aren't supposed to have an application manifest. If they do, the linker won't be able to resolve which application manifest should be used when we are building an assembly from multiple netmodules
3. If one netmodule is dependent on another netmodule, a reference to the dependent module should be specified using the AddModules XML node in the .proj file. Similar to DLLs, netmodules cannot be circularly referenced

Once the .proj file has been modified, the project can be built using msbuild. If the project is a part of a solution, building the solution will create a netmodule for this project - msbuild solution_path

Once the netmodules are created using msbuild, a DLL can be created from them using the Assembly Linker of the .NET framework - al /t:library /out:"path_to_dll" path_to_netmodules

With this we have a DLL which contains all the netmodules bundled into one. To understand this better, let's consider an example - I have three projects, one in F#, VB.Net and in C# under a solution called MultiLanguage.

The .fsproj, .vbproj and .csproj have to be changed as mentioned above. From the netmodules obtained from msbuild, the DLL can be created using the assembly linker.

The sample solution and the build script can be downloaded from here. I automated the .proj changes using Microsoft's Build Engine API. You can give it a try here; it takes the .sln file as an input and modifies all the projects in the solution to generated netmodules.

Windows Phone 7 - Hello World!

Ever since Nokia announced its partnership with Microsoft and made Windows Phone 7 its primary smartphone operating system, I anxiously wanted to give it a spin. So in this post, I am going to start by setting up a development environment for Windows Phone 7 followed by a simple Hello World example.

Windows Phone 7 is primarily a successor of the Windows Mobile Platform (the last one in the line being Windows Mobile 6.5). It's a major revamp in Microsoft's Mobile strategy in terms of its application development model.

Windows Phone 7 no longer supports unmanaged code (Win32 and C++ are gone). It is purely built on Managed Code covering Silverlight, XNA and the .NET Framework (Programming Languages include C# and VB.Net). Here's a nice article on Windows Phone 7 application development model - http://blogs.msdn.com/b/abhinaba/archive/2010/03/13/windows-phone-7-series-programming-model.aspx.

To get started with Windows Phone 7 development, download the Windows Phone Developer Tools RTW (Release To Web). It includes the following tools -

1. Visual Studio 2010 Express for Windows Phone
2. Windows Phone Emulator Resources
3. Silverlight 4 Tools for Visual Studio
4. XNA Game Studio 4.0
5. Microsoft Expression Blend for Windows Phone

If you are already using a higher version of Visual Studio or Expression Studio, this toolkit will install extensions to the existing IDEs. The release is also available as an .iso image. Though the Windows Phone 7 toolkit comes equipped with Expression Blend, for most of the basic applications the design mode of Visual Studio should be sufficient.

To get a hands-on experience with Windows Phone 7, let's take a simple application which accepts a username and greets him/her in the next page. The best part of Windows Phone 7 development is that it is pretty much in the lines of other Visual Studio project types we are familiar with like the Windows Forms Applications, WPF Applications and ASP.Net Applications where in the UI can be created easily using the designer and the business logic is maintained in a code beside event driven model in another .cs or .vb file depending on the programming language.

As already mentioned, Windows Phone 7 is built on Silverlight and thereby every screen has a corresponding XAML (eXtensible Markup Language) page, each having its own code file. Movement between these XAML pages is made possible through an instance of a NavigationService class for every page.

There are three important methods provided by NavigationService - Navigate, GoForward and GoBack. Navigate takes a Uri instance specifying the URI location and loads the XAML page specified.

NavigationService.Navigate(new Uri("/NamePage.xaml", UriKind.Relative));

GoBack and GoForward loads the previous and next pages respectively.

NavigationService.GoBack();
NavigationService..GoForward();

Parameters can be passed across XAML pages by appending them to the URL and retrieving them using the QueryString property of the NavigationContext object. This concept of Windows Phone 7 was very surprising. Since Mobile Applications are very similar to Windows Applications, passing parameters through URLs like Web Applications looked a bit off-track.

NavigationService.Navigate(new Uri("/NamePage.xaml?name=" + name, UriKind.Relative));
string name = NavigationContext.QueryString["name"];

Download the example code here.

Introducing ADO.NET Entity Framework

This is the third and final part of my ORM series in which I am going to introduce the ADO.NET Entity Framework, an in-built Object Relational Mapping model of the .NET Framework.

Similar to the previous post, this one also covers the same four principles -

• Configuring a .NET project for the ADO.NET Entity Framework
• Inserting data from Objects directly
• Retrieving data using Object Lists an LINQ
• Changing the Database Management System

Though the screen-shots and the example codes emphasize on C#, the principles are same for all the .NET languages.

Requirements to run the example in this article - Visual C# Express, SQL Server Express and MySQL. Download Visual C# Express here, SQL Server Express here and MySQL here.

Configuring a .NET project for the ADO.NET Entity Framework

To configure a .NET project to work with the ADO.NET Entity Framework, an Entity Data Model is added to the project. The Entity Data Model is primarily a schematic representation of the database tables stored as a XML file. Each of these tables is converted into a class and foreign key relationships between these tables are maintained as Lists inside the objects.

Visual Studio provides a Wizard to create Entity Data Models. Right click on the project and select New Item from the Add menu. Choose the ADO.NET Entity Data Model template from the chooser. There are two approaches available to build the Entity Data Model - Database First Approach and the Model First Approach.

In the Database First Approach, the Entity Data Model is generated from existing table structures. The wizard allows the developer to setup a connection by providing the Database Server name and the Database name. The wizard also allows developers to choose tables, views and stored procedure that are to be a part of the Entity Data Model.

In the Model First Approach, the Entity Data Model is created from scratch and the database tables are generated based on this model.

Once the wizard completes, a designer opens up which shows a schematic representation of the generated .edmx file. This file contains the table mappings as a XML along with a code behind file with a .Designder.cs extension for the auto-generated classes corresponding to the tables. However a drawback with the Entity Data Model is that it combines all the classes into a single file which makes manual maintainence a little difficult.

I used the Database First approach to create the sample application to insert and retrieve data. You can get the MySQL and SQL Server scripts along with the Visual Studio solution here.

Inserting data from Objects directly

The Entity Model generates a class which inherits from ObjectContext. This class acts as a data manager to connect to the Database Management System to retrieve, insert, update and delete records.

ADO.NET Entity Framework maintains the records of the table as a List of objects. To insert new records into the table, create new objects, add them to the appropriate lists and then save the changes using a ObjectContext instance. The following piece of code shows how objects can be persisted -

using (Context context = new Context())
{
// Create the object
context.Objects.Add(object);
context.SaveChanges();
}

ADO.NET Entity Framework does a wonderful job while storing objects with foreign key dependencies. These dependencies are maintained using lists as objects and while storing these records, the appropriate identity keys are inserted into the child tables. However this feature is limited to a few DBMS like SQL Server.

Retrieving data using LINQ and Object Lists

The ADO.NET Entity Framework retrieves data from the back-end tables in the form of object lists. So accessing records is as simple as iterating through these lists.

using (Context context = new Context())
{
foreach (Object object in context.Objects)
// use the object appropriately
}

Since data retrieval is in the form of lists, developers can piggy-back on an other .NET framework feature - LINQ (Language Integrated Query). LINQ makes conditional querying of data a lot easier.

using (Context context = new Context())
{
var objects = from object in context.Objects where "condition" select object;
// use the objects
}

Changing the Database Management System

Before changing the DBMS of the Entity Model, it is important to understand how the ADO.NET Entity Framework stores the connection strings and the mapping between classes and the back-end tables. The connection string is stored in the App.Config file of the project and the table mappings are stored as a XML in the form of the .edmx file as mentioned earlier.

Unfortunately the ADO.NET Entity Framework varies it's implementation with the DBMS. Because of this modifying the XML manually isn't easy. For Example, ADO.NET Entity Framework does not support foreign key constraints in the form of lists for DBMS like MySQL.

The sample application contains another Entity Model which connects to a MySQL Server containing similar tables. To use MySQL with the ADO.NET Entity Framework, an connector is needed. The MySQL Connector/NET is available here.

Introducing Hibernate In Java Using NetBeans

In one of my recent posts, I introduced the theoretical topic of Object-Relational Mapping (ORM) - http://gautam-m.blogspot.com/2010/11/object-relation-mapping.html. In this post I am going to take a step forward and introduce Hibernate - an open source Java persistence framework from JBoss.

This post covers four basic principles of Hibernate -

• Configuring a Java project for Hibernate
• Inserting data using Object Persistence
• Retrieving data using Hibernate Query Language (HQL)
• Changing the database configuration to connect to another DBMS

Though the post and screen-shots emphasize on NetBeans, the concept is the same for all IDEs. Hibernate configuration files can definitely be written without an IDE but make sure all the required class libraries are properly referenced.

Requirements to run the example in the article - NetBeans, MySQL, JavaDB, Java and Hibernate. Java can be downloaded here, installing the All NetBeans package will cover JavaDB and Hibernate. and MySQL can be downloaded here.

Configuring a Java project for Hibernate

The crux of Hibernate is the creation and usage of configuration files. There are three types of configuration files which are to be setup for Hibernate -

1. The .cfg.xml file - this is the main configuration file which contains information about the database like the database URL, the driver, the username and password, etc. Hibernate can optimize it's behavior depending on the DBMS being used. To facilitate this, a property called Dialect is specified. However this is an optional property as Hibernate can deduce this depending on the JDBC metadata returned by the driver
2. The .reveng.xml file - this file holds the data corresponding to the schemas and tables being utilized by Hibernate in the application
3. The .hbm.xml - these files maps POJOs (Plain Old Java Objects) to the table schemas of the database

Typically one .cfg.xml and one .reveng.xml exist for a project and one .hbm.xml exists for each table (mapped to a class). The .hbm.xml maps the object properties to the table columns. It is possible to add new properties to the class which have no effect on the backend tables.

To create these files in a NetBeans project, select New File and select the following File Types from the Hibernate category -

1. Hibernate Configuration Wizard
2. Hibernate Reverse Engineering Wizard
3. Hibernate Mapping Files and POJOs from Database

Follow the wizards to complete the configuration setup. I used MySQL and JavaDB as my DBMS to create a sample application to insert and retrieve data. You can get the MySQL and JavaDB scripts along with the NetBeans project here.

Inserting data using Object Persistence

Inserting data is a cake-walk in Hibernate. All that is there to do is to create the object and store the object data in the database tables using the save method of a SessionFactory object. The following piece of code persists the object data -

SessionFactory sessionFactory = new Configuration().configure().buildSessionFactory();
Session session = sessionFactory.openSession();

// Create the object
session.save(object);

Retrieving data using Hibernate Query Language (HQL)

Retrieving data is done through a query language designed for Hibernate called the Hibernate Query Language. HQL is a Object-Oriented Query Language and is very much similar to the traditional SQL we use. The beauty of HQL is that the result of the query is returned as a list of objects rather than as a ResultSet. These objects can be used directly in the code without any overheads. HQL is very wide topic, so I am going to skip the details here but there are several tutorials available for HQL on the Internet. The createQuery method of the above Session object is used along with the list method of the Query object to get the objects -

Query query = session.createQuery(queryString);
for (Object object : query.list()) {
// cast and use the object appropriately
}

Changing the database configuration to connect to another DBMS

The best feature of Hibernate according to me is it's ability to change a DBMS without any change to the application code. To change the DBMS, open the Hibernate Configuration File (typically hibernate.cfg.xml) and change the dialect, driver class, connection URL, username and password to the values corresponding to the new DBMS.

These changes can be done either through the design view or directly on the XML.