Rooting & Unrooting Micromax Funbook Pro

Well, here is another rooting guide from me. This time it's on Funbook Pro P500, a 10 inch tablet from the Indian manufacturer Micromax. I am not going to talk much about the tablet here, there are several reviews on it across the internet and a good Google search could land several results on this. All I can say about it is that I have been using it over the past 6-7 months and it has been an awesome tablet at its price point.

Before I started writing this blogpost I noticed that there are several articles on the internet emphasizing on rooting Funbook Pro, but most of them focus of either using an application or a tool to get this done. However in this post, I am going to use only the Android Debug Bridge (ADB) keeping you in control of the process.

In one of my previous posts (Rooting & Unrooting HCL Me X1), I have already mentioned the advantages and risks of Rooting, please do go through it before proceeding further. Let me reiterate one point from that post - Trying to root your device might brick the device. I am not responsible if you brick your device, so proceed at your own risk.

Rooting the Device

You don't really require any exploit or any application to root the Funbook Pro, it is possible to root the device using the adb shell so make sure you have the Android Debug Bridge (ADB) on your system.

Connect the device through the USB and open the shell using adb shell at the command prompt. You would be seeing a $ here indicating that the shell does not have root access. Exit from the adb shell and restart the adb on the device using adb root, a message stating that restarting adbd as root would appear. Go back to adb shell, you would see a # now indicating that the shell has root access.

Inspecting the ro.secure property either using getprop ro.secure or from /default.prop would return a value of 1. This indicates that the adb will not login as root by default and hence we have to use the adb root command when we want root access using the adb shell.

Though the adb shell has root access, the su command which is used in Linux doesn't work in Android since there is no su binary in Android. So our next step is to setup the su command on the device. Download the su binary and the Superuser application from here.

There are two steps for rooting the device -

1. Copying the su binary to the /system/bin directory

Unlike most other devices, the /system file system in Micromax Funbook Pro isn't read-only making things simpler for us. Use adb push and copy the su binary to the device with the command adb push su /system/bin. Don't forget to set the permissions of the binary using chmod 4777 su.

2. Install the Superuser.apk from the system/app directory of the zip

Install the application using adb install or transfer the file to the sdcard and install it from there. The Superuser application intercepts the calls made to the su binary and informs the user about the usage so that unauthorized applications cannot attain root access.

Unrooting the Device

To unroot the device, uninstall the Superuser.apk and delete the su binary from the /system/bin directory.

jQuery getScript, FireBug And Caching

Recently in one of my projects I had a requirement of loading JavaScript files dynamically. It was then I took a look at the jQuery getScript method. At first it looked perfect for my requirement, however as I started using it for my project I realized there were two drawbacks with its default implementation.

1. There isn't any caching for the scripts loaded by getScript
2. Debugging these scripts in FireBug is quite difficult since FireBug cannot map the script code to the script filename

In this post, I am going emphasize on my experiences with getScript and how I got both it to work with both these requirements. I am sure people who are spending time to get these features working will find this useful.

To help me with this, I put up an example Visual Studio project available here. I used Visual Studio for comfort but any text editor will work since it is all about HTML and JavaScript.

Basic Example

Well, getScript is pretty easy to use. In its simplest form, all we have to do is provide it with the script name. Here's an example on how it looks.

But the interesting point is the way it works. Here's a screenshot in FireBug which show three calls being made to the server since getScript appends a random number to the server call causing the browser cache to fail. And the script debugger is also unable to recognize the script name.

Enabling Caching

Solving the caching issue was simple. Following the recommendation on the jQuery getScript documentation page, I setup AJAX caching using ajaxSetup as show below.

This worked well but the only problem that I noticed was that ajaxSetup was a global option and setting it there would make all AJAX calls cached which could mean trouble for the regular data calls we make. Fortunately the documentation page also had a solution for this. I wrote a new jQuery method using that solution as shown below.

Making getScript FireBug Compatible

At first I had had no clue on why this was happening. But thanks to the guys at TechMonks who wrote an absolutely brilliant article on this problem, I could get this fixed. In short the problem was because of how jQuery implements getScript wherein it uses AJAX to the script content and makes it inline. Do read the entire article here for more details.

Here is a code snippet which make getScript compatible with FireBug. The reason why this works is because it loads the JavaScripts files as external resources instead of making it inline and FireBug shows them in the debugging panel.

Internal Caching

The new firebugGetScript from TechMonks works great with FireBug and also facilitates browser caching. However I had one more issue with it. My web application was completely dynamic and had to load a lot of scripts on the fly. Because of this there were scenarios where I was calling the same script through firebugGetScript. However the method wasn't smart enough to recognize this and added the script element multiple times.

I added the ability to recognize this and changed the code to the following.

And things looked perfect with this. I have been using this for quite some time in two different web applications and it's working as expected.

That's it for this post, hope the next time isn't as far away as this ;)

Dynamic Expression Evaluation In .NET

Many applications need to evaluate expressions at run-time. This can be done using a dynamic expression evaluator. Examples of such applications are data analysis applications that perform calculations from user-defined expressions, spreadsheet software like Excel which allows users to define formulae, etc.

There are several ways of doing it in .NET, each one having its own pros and cons. This post covers the various approaches and at the end compares their performance along with code examples for each one of them. These approaches can be categorized as follows

Parsing

One of the most common ways of achieving this is by writing a parser. The parser can parse the input expression, generate data structures and eventually evaluate the sub-expressions to find the result of the expression.

Developers can either write their parser or use one of the already available parsers. The biggest challenge with developers writing their own parser is that building a parser is quite complicated and requires a lot of coding effort. The advantage of using parsers for expression evaluation is that it doesn't require any external processes and the implementation can be pure managed .NET code.

Here is an expression evaluator from Code Project written in 100% Managed .NET. As the article points out, the expression goes through various phases like tokenization, parsing and interpretation before the final result is obtained. Fast Lightweight Expression Evaluator (Flee) is another expression evaluator from CodePlex. It uses a custom compiler, strongly-typed expression language and lightweight codegen to compile expressions directly to IL.

Runtime Compilation

Instead of building a custom parser, another alternative is using the .NET compilers. They can parse expressions in all the .NET languages and generate IL code. The best thing about compiled code is that execution of the code is quite fast but on the other hand it takes a significant time to compile the code.

The .NET framework includes a mechanism called the Code Document Object Model (CodeDOM) for this requirement. The System.CodeDom.Compiler namespace defines types for generating source code from CodeDOM graphs and managing the compilation of source code in supported languages. The .NET framework includes code generators and code compilers for C#, JScript and Visual Basic. Here is a tutorial on how CodeDom can be used for compiling C# code.

CodeDOM uses the compiler to compile the code, so it expects the compiler to exist on the clients system. The problem with CodeDom is that a new assembly is created and loaded into the AppDomain; and since we cannot unload an assembly in .NET, we end up with several unnecessary assemblies in memory. There are two solutions for this

• Another temporary AppDomain can be created where the assemblies are loaded and the AppDomain can be disposed at a later time
• Using DynamicMethod to create a method at runtime. The IL code for this can be obtained from the CodeDOM. Here is a tutorial for this

The Eval Function

Eval is a function which evaluates a string as though it was an expression and returns a result. Most of the languages that support eval are scripting languages like JavaScript, Python, etc. This is because most of the scripting languages are interpreted, not compiled. This provides these languages enormous capability to evaluate dynamic expressions since an eval call is nothing but another line to interpret at runtime. Compiled languages like Java and the .NET languages (like C# and VB.NET) don't have this flexibility since they have to generate IL/Byte code before they run. However some of the .NET language do support eval even though they are compiled.

JScript.NET is Microsoft's implementation of ECMAScript in the .NET initiative. JScript.NET contains an eval function which can evaluate dynamic expression. It compiles the code every time eval called and then runs it, so there definitely is a performance hit. The downside of eval has been explained in this article.

IronPython and IronRuby are the .NET implementations of the Python and Ruby scripting languages. These are dynamic scripting languages unlike C# and VB.NET which run on the Dynamic Language Runtime. With the DLR, several dynamic constructs like eval could be implemented and since it is based on .NET, it is interoperable with C# and VB.NET. Here is a tutorial on embedding IronPython in C#.

Microsoft Roslyn

Microsoft Roslyn is Microsoft's biggest change in compilers since the advent of .NET. However this is currently in CTP and isn't available in .NET 4.0. Roslyn provides the compiler as a service (library). This is better than CodeDOM since it won't create new processes. Roslyn has wonderful features especially for Dynamic Compilations, Code Reviewers, etc. Here is a tutorial on using Roslyn with C#.

The performance benchmarks of these approaches are as follows. The code used to obtain these benchmarks is available here.

Backup & Restore HCL Me X1

If you are planning to experiment with any Android phone/tablet, the first thing you would like to do is take a backup of it just in case things get messy. Typically in such situations you would like to take a backup of not only the user-data but also the operating system.

In this post I will be discussing on how to create a backup and restore the backup. You don't need to root your device for this since adb already has root access in HCL Me X1. But you will need fastboot which can be obtained by compiling the Android source code.

Remember that the source code can be built only on a Linux distribution, so make sure you have one (I would recommend Ubuntu 11.10). fastboot will be available at <source_code_dir>/out/host/linux-x86/bin/fastboot after the compilation. If you don't want to compile the source, you can try a compiled version of fastboot from the link specified at the end of the post; I built it on Ubuntu 11.10.

Replacing the Recovery Image

Before replacing the recovery image, create a backup of it in case you want to revert it back. Open the shell of the device using adb shell. Find the mount-point of the recovery image using cat /proc/mtd. There will be an entry like - mtd6: 00a00000 00200000 "recovery"

The recovery image can be created by reading the contents of the recovery mount-point and writing it into a file. This can be done using the following command - cat /dev/mtd/mtd6 > /sdcard/recovery.img

The recovery image is written to the sdcard in this case, you can redirect it anywhere you want to. The recovery image can be flashed back using fastboot. For this you have to boot the device into the bootloader using adb reboot-bootloader. The image can be flashed using the following command - fastboot flash recovery recovery.img

You can backup other mount points like system, userdata, etc. using this method but it's a lot easier to do so using the ClockworkMod's Recovery Image. You can download it from the link specified at the end of the post. Flash it using the same fastboot command as above.

Creating a Backup

Reboot the device into the ClockworkMod's recovery mode using adb reboot recovery. This console has options to backup and restore the operating system. However it does this using the sdcard, so make sure you have one in the device before you proceed.

From the recovery console, select Backup and Restore > Backup. After a series of messages the ROM will be backed up.

Restoring a Backup

From the recovery console, select Backup and Restore > Restore. Select the backup you want to restore. After a series of messages the ROM will be restored.

In case you are looking for the stock ROM you can find it here. It's not the exact stock ROM but an unrooted factory reseted ROM. And the original recovery image, ClockworkMod's recovery image and the compiled version of fastboot is available here.

Rooting & Unrooting HCL Me X1

Android is primarily an operating system based on the Linux kernel. So similar to other Linux based operating systems Android has a special user account known as the root or super-user. Unlike other user accounts which have limited control over the operating system this account has complete access to the system.

However several (almost every) carriers and manufacturers don't allow users to access this account mainly due to security concerns. Few carriers and manufacturers also do this because they don't want users to access features for free when they can sell them :). Rooting is a process that allows users of phones/tablets to gain this privileged control (the root access). Here is an article with the Top 10 Reasons To Root Your Android Phone.

Though rooting your device is not illegal it might void the warranty of the device. So think twice (make that trice) before rooting your device and make sure you have a good reason to do so. Read these articles on The dangers of rooting your Android phone and What is Rooting on Android? The Advantages and Disadvantages

The process of rooting a device changes from device to device. It usually involves using an exploit to either gain temporary root access or to flash a custom recovery image to attain permanent root access. There are popular exploits like GingerBreak, psneuter, etc. and applications like GingerBreak.apk, SuperOneClick, etc. for rooting a device.

If you have read the articles and still want to go ahead let me remind you once again that trying to root your device might brick the device. I am not responsible if you brick your device, so proceed at your own risk.

Rooting the Device

You don't require any exploit or any application to root HCL Me X1 and in-fact vulnerabilities like GingerBreak have already been patched. However it is possible to root the device using the adb shell so make sure you have the Android Debug Bridge (ADB) on your system.

Connect the device through the USB and open the shell using adb shell at the command prompt. If you are seeing the # here then that indicates that your adb shell has root access. Instead if you are seeing a $ this process won't work for your device. Please put in a comment if you see $ so that me and the other readers know that HCL has changed something.

Typically in most of the devices the adb shell does not have root access. However, looks like the Android build of Me X1 isn't a production build. Not sure if this was done knowingly or unknowingly by HCL but either way the tablet has root access through the adb shell. You can confirm this by trying one of these -

• Type adb root at the command prompt, it will show you a message stating that adbd is already running as root
• Type getprop ro.secure at the adb shell, it shows 0
• The ro.secure property can also be examined from the /default.prop. If this is 0 then it indicates that the adb will login as root by default. Note that this file cannot be changed since it's a part of the ramdisk

Though the adb shell has root access, the su command which is used in Linux doesn't work in Android since there is no su binary in Android. Download the su binary and the Superuser application from here (backup).

There are two steps for rooting the device -

1. Copying the su binary to the /system/bin directory

Since this directory is on the Android PATH, applications will be able to call su directly. However since the /system is a read-only file system (type mount at the adb shell to check this), the file cannot be directly copied. Push the su binary from the system/bin directory in the zip to a temporary folder of the device (like /data/local/tmp) using adb push. However note that the su binary does not work directly from this directory since applications on the /data file system do not have the setuid bit on. Remount the /system with read-write capabilities with the mount -o remount -rw -t yaffs2 /dev/block/mtdblock2 /system command (check the actual device mount-point using mount). Copy the su binary to /system/bin and set the permissions of the binary using chmod 4777 su.

2. Install the Superuser.apk from the system/app directory of the zip

Install the application using adb install or transfer the file to the sdcard and install it from there. The Superuser application intercepts the calls made to the su binary and informs the user about the usage so that unauthorized applications cannot attain root access.

Unrooting the Device

To unroot the device, uninstall the Superuser.apk and delete the su binary from the /system/bin directory.

A Primer To Android Development

As most of you know, Android is a mobile operating system initially developed by Android Inc. and presently being developed by the Open Handset Alliance led by Google. Android consists of a modified Linux kernel and software built on the Apache Harmony framework (an open source Java implementation). The best thing I like about the Android platform is its versioning. The code names are released in alphabetic order of dessert items - Cupcake, Donut, Éclair, Froyo, Gingerbread, Ice Cream Sandwich and Jelly Bean.

I am a newbie myself when it comes to Android, so the past few days I was trying to understand the various development models of Android. During my search I came across several good tutorials. This blog post is primarily to consolidate these tutorials for beginners like me.

Android, like most open source Operating Systems allows developers to look at development from two different perspectives - working on Android (Application development) and working with Android (Operating System development, not possible with proprietary Operating Systems). Google understands this and therefore has two different sites for Android developers - Android Developers (has the Android SDK to build applications) and Android Open Source Project (has the Android source code).

Android Developers

The Android Developers site provides various Development Kits to create applications for the Android platform. The Android SDK allows developers to write applications for the Android platform in a slightly modified version of Java which runs on a virtual machine called Dalvik.

The Android SDK can be installed by following the instructions specified here - Installing the SDK. And in case you want to start programming right away, here's a classic Hello World tutorial.

The Java APIs are pretty much what most developers use. However there might be situations where you would like to harness the true potential of the underlying Linux kernel through a low level language like C. Android does allow developers to do that but they cannot call the C binary directly as an application, they will need to go through Java with the JNI (Java Native Interface). This methodology of development is supported by the NDK (Native Development Kit).

The Android NDK can be installed by following the instructions specified here - Installing the NDK. The NDK does come with the Hello World example but it isn't as easy as the regular one.

The most useful tool Android provides developers with is the Android Debug Bridge. It is a command line utility that supports several commands like pulling/pushing files to and fro the device, installing applications, etc. This is one tool that every Android developer will have to use irrespective of whether he is developing applications or playing with the source code. Please note that the adb is part of "Platform Tools" which can be installed through the Android SDK Manager. If you stuck trying to install adb, read this.

Android Open Source Project

The Android Open Source Project site provides access to the source code of Android. A majority of the operating system infrastructure of Android is distributed under the Apache License and the core kernel is licensed under the GPLv2 License.

Certain developers use the Android Open Source Project to find vulnerabilities in the Android operating system and exploit them to execute tasks which cannot be done normally like attaining root access. Examples of these exploits are GingerBreak, psneuter, etc. Typically these vulnerabilities are patched for security reasons.

A majority of developers use the Android Open Source Project to modify and customize the operating system to create custom images. Several vendors add device drivers for their hardware and install pre-release applications to the operating system before flashing it onto the device. There have also been situations wherein vendors have customized the UI to suit their product lines before flashing.

You can obtain the complete source of the Android operating system by Downloading the Source Tree. There is an awesome tutorial on building Android 2.3.7 on Ubuntu 11.10 (64 bit) here. Even though the AOSP site specifies instructions on compiling the code, they are a bit outdated so visit the above link if you are stuck anywhere.

Underneath the layers of application frameworks, Android is primarily a Linux kernel and therefore is built on a C library. Unlike the traditional Linux based operating systems, Android uses a custom tiny libc called bionic. C programs can be written in an Linux environment running on x86/x64 architectures and cross compiled to Android running on ARM by linking Bionic to gcc. Here is an excellent tutorial which shows you exactly that. The compiled code can be run using adb shell. However note that codes compiled this way aren't recommended to be used with Java applications, the NDK should be used instead.

HCL Me X1 - A Customers Review

HCL Me X1 is HCL Technologies latest tablet and is one of the cheapest tablets available in India priced at Rs. 10,490. HCL released the X1 on October 14th. This is HCLs third tablet in the Me Series after the AE7-A1 and AM7-A1. I ordered the tablet on 19th through the HCL Store and received it this Tuesday. The entire week I have been toying with the tablet and here are my initial impressions on the Me X1.

This is my first Android tablet (my first Android device in fact) and one of the main reasons I bought the tab was to put up with the Android craze and have my share of fun with the Android platform. So expect to see several posts on the Android platform and the HCL Me X1 on this blog :).

I have been researching Android budget tablets for quite some time and finally decided to get myself the Me X1 (The Lenovo IdeaPad A1 was another option but decided to go with Me X1 after Lenovo priced it at Rs. 15000). Before going deeper into the review let me start by saying that the HCL Me X1 is a good tablet from a price to performance perspective and does a decent job in gaming, music and movies.

The specifications of the HCL Me X1 can be found here. Though HCL hasn't specified the actual Application Processor they are using, it is expected to be the Marvell ARMADA 610.

As evident from the above specifications, the HCL Me X1 doesn't have a SIM card slot and so doesn't support 3G directly. However it does support 3G through a USB dongle. This wasn't much of an issue for me since I was planning to use my Nokia X6 as a WiFi hotspot (haven't tried this yet). The touch responsiveness of the device is good and Gingerbread performs well on the device thanks to the 1 GHz processor.

However the HCL Me X1 does have its shortcomings,

• A major drawback of the Me X1 is that it doesn't have the Android Market on it. Whatever HCL may say, its App Store is no match to the Android Market
• The device works for around 6-8 hours (usage + standby) with a single charge. I wasn't satisfied with this as my Nokia X6 gives me much more than this. However benchmarking a tablet to a phone isn't correct
• The display would have been better if it was a slightly brighter

The biggest weakness of HCL is their delivery service. When most online sites ship within 2-3 days, HCL as a policy takes a minimum of 7 to 10 days to ship the product. However they sent me the devices in 7 days when I pestered them a lot. In fact I would recommend people to buy the device at a local Sangeetha store where you can get it immediately. On the other hand their customer care was good, especially the Live Chat, they answered a lot of questions and followed up on phone as well.

I think the biggest technical improvement opportunities the Me X1 has are the integration of the Android Market and getting out an Ice Cream Sandwich as soon as possible. This should attract more customers towards the product.

Before I wrap up, here are some excellent reviews on the tablet which helped me take a decision on the purchase -

• Arpan Deb's review on the HCL Me X1
• iGyaan's video review on the HCL Me X1
• HCL Me X1 - To Buy or Not To Buy

Update 2011/11/10

After several discussions at arpandeb, we finally concluded that the actual Application Processor being used is the Telechips TCC 88xx. We also found striking similarities between HCL Me X1 and Coby Kyros MID7022.

Named Groups In Regular Expressions - Java 7

One of the new features added in Java 7 was the introduction of named groups in regular expressions through the java.util.regex package. This article covers the various features of named groups in regular expressions and their syntax in Java.

Parts of a Regular Expression can be grouped together by placing them inside round brackets. A major advantage of grouping is that various regex operators can be applied to these groups. Grouping is also useful for back-referencing a match. This allows developers to write regular expressions involving complex repetition patterns more easily.

The value of the captured group can be retrieved directly or be referenced in replacement patterns for changing the format of an input string. Though Java supported Regular Expressions and Grouping through the java.util.regex package from Java 1.4, it was limited to numbered groups.

Here's an example showing how numbered groups can be used in Java. To read more about Grouping and Backreferences, check this out.

Parallel to numbered groups exists named groups. Named groups do not change the underlying concept of grouping and back-referencing but provide more readability to the regular expressions and make back-referencing easier for developers since remembering names is easier than remembering the relative position of a group.

Named groups are already present in languages like Perl, Python and .NET but wasn't available to Java prior to Java 7. Well, better late than never.

The newly added RegEx constructs in Java 7 are as follows -

• (?) defines a named group "name"
• \\k back-references a named group "name"
• ${name} can be used to reference a captured group in a replacement string
• group(String name) returns the value of the captured group

Here's an example showing how named groups can be used in Java 7. To read more about Named Groups, check this out.

However if you are using Java 7, I would strongly recommend using the java.util.regex package but if you are using a Java version older than 7, there are other alternatives available for named groups in Java like Named-RegExp and JRegex. These alternatives were highly recommended by other developers at stack overflow.

DC Comics Reboots The Universe

Today's a big day in the DC Universe. For people wondering what DC Comics is - DC Comics Inc. is a subsidiary of Warner Bros. and is one of the largest and most successful companies operating in the comic books industry. It introduced a majority of the popular superheroes we have grown up with like Superman, Batman, Green Lantern, Catwoman, etc.

Effective this month, their entire comic series has taken a major overhaul and all DC Universe titles being cancelled or restarted at #1. Titles which have been around for more than 70 years like Action Comics and Detective Comics, each having around 900 issues credited to them are also being rebooted making it a very big thing in the comic world.

The DC Comics Reboot (AKA The New 52) has been termed as the biggest thing to happen to comic books since Crisis On Infinite Earths (which happened 25 years ago, before I was born :P). This revamp is going to affect not only the storylines but also the way books are being marketed.

Things are changing a lot in the comic world - print is dying, comic book sales are going down year after year and the revenue streams for comic book writers are going dry. With movies like The Dark Knight, X-Men, Superman, etc. making big impressions at the box office, their sources of inspiration going down would be seem very strange but it's true.

I personally would attribute to two major reasons -

• Complex Story Lines - Movies aren't driving comic sales since movies are far far behind the comic series. Today a newbie can understand The Dark Knight or play Batman - Arkham Asylum but when they grab a copy of Batman comic off the shelf they seem out of sync
• Digital Media - Complex Story Lines are a problem but a far bigger problem is the influence of digital media on the world. People today are far more addicted to sites like Facebook, Twitter, Youtube, etc. and are hooked to devices like iPads, Nooks, Kindles, etc. than reading a physical book. Gone are the good old days when we visited a store to grab a book; we today either download an ebook or wait the movie to come out ;). I don't blame anyone but this is price which we have to pay for embracing technology. The bigger point is that organizations have to understand this and adapt to it

The DC Reboot primarily addresses both these issues; a reboot means that things start from beginning and today's generation can understand the storyline from scratch without having to dig back for older books. On the darker side, this will certainly have a mixed reaction from the existing collector base which has to be dealt carefully.

Also every DC Comics book from here on out will be digitally distributed "day and date". This means that on the same day individual issues hit the shelves of the comic stores, they will also be available to read digitally. The digital distribution of the comic books is something which I liked a lot but I can foresee a major problem of piracy :(. But still it's a great move and if the pricing is maintained properly for people around the globe, it can create a huge impact for the comic world.

For those of you wondering about character origins and histories, it's still unclear what's going to happen - a few of them may be left untouched while the others may see a major change. For instance I didn't see the Martian Manhunter in the Justice League #1.

This move will definitely stir up some hype and I can't wait to see how Marvel responds to this :). Either way, this year's Comic Cons are going to have a lot to talk about and this move will surely have an impact on the future of the comic book industry.

Check out the trailer of DC 52 below -

Check out these cool articles on the DC Reboot -

• DC Comics’ New 52 Reboot - The Complete Guide
• DC Comics hopes revamped heroes and digital will save the day

Remote Debugging Java Applications

After my previous article about Remote Debugging .NET Applications using Visual Studio 2010, I was curious to see how Remote Debugging works for Java Applications. This blog post covers Remote Debugging Java Applications using NetBeans 7.0.

The basic concepts of Remote Debugging are the same for .NET and Java but the process of setting up the host and remote computers varies between Visual Studio and NetBeans. Similar to the .NET application used in the previous post, the screen shots correspond to a simple Java application which would popup a MessageDialog on a button click. Get the code here.

Before configuring the host and remote computers, it is vital to understand the Java Platform Debugger Architecture (JPDA). JPDA provides the infrastructure you need to build end-user debugger applications for the Java Platform. It includes the following APIs broken into three layers -

• Java Debug Interface (JDI), a high-level Java programming language interface including support for remote debugging
• Java Debug Wire Protocol (JDWP), which defines the format of information and requests transferred between the process being debugged and the debugger front end
• JVM Tools Interface (JVM TI), which is a low-level native interface that defines the services a JVM provides for tools such debuggers and profilers

Configuring the Remote Computer

Run the Java application using the -Xdebug and -Xrunjdwp options from the command line.

java -Xdebug -Xrunjdwp:transport=dt_socket,address=6000,server=y -jar RemoteDebugging.jar

Here's a description of the options of the java command -

Option	Description
-Xdebug	Enables debugging support in the VM
-Xrunjdwp	Loads in-process debugging libraries and specifies the kind of connection to be made

The -Xrunjdwp option has several sub options. Here are the descriptions of the ones that are used above -

Option	Description
transport	Name of the transport to user in connecting to debugger application
address	Transport address for the connection If server=n, attempt to attach to debugger application at this address If server=y, listen for a connection at this address
server	If y, listen for a debugger application to attach If n, attach to the debugger application at the specified address

By default the application starts in suspended mode. In suspended mode the application waits for a debugger to attach itself to the server at the specified port before the application starts.

Configuring the Host Computer

The host computer is the system running NetBeans 7.0. Open the code of the application in NetBeans and select "Attach Debugger". Specify the Connector as SocketAttach, the Host as the hostname of the remote system and the Transport and Port as specified above.

Specify the breakpoints in the code and they would hit appropriately. There are two major bottlenecks in Remote Debugging -

• The code from which the executable was built should be available at the time of debugging
• Applications cannot be configured for Remote Debugging at runtime. The -Xdebug option must be specified at the instantiation of the application, making debugging live production code difficult

Before I conclude, here's an article from OTN (Oracle Technology Network) on the Java Platform Debugger Architecture. Visit it if you would like a deeper insight into Java Debugging. The schematic of the JPDA is from a weblog, check it out here.

Remote Debugging .NET Applications

From the past few days I have been stuck in resolving a bug which cropped up in one of my applications after it went to Production. The worst part was that it was a machine-specific issue and we couldn't reproduce it in any of our development systems. While trying to find a solution for this, I came across the concept of Remote Debugging.

Remote debugging is debugging a remotely running application through a development environment running on a system other than the one running the app. This is done by connecting the remote system to the system containing the development environment (in turn the debugger) through Sockets. Theoretically this is achieved in two steps -

• The remote computer would open a socket and listen to debug instructions through it. These instructions are feed to the application that is being debugged and the application responds appropriately
• The debugger would connect to the socket opened by the remote system and send instructions through it

Remote debugging can be used in many live situations. Examples include -

• Debugging a machine specific issue which can't be reproduced on systems running the development environment (my exact requirement :))
• Debugging applications which can run only on the server machine due to dependent third party libraries that cannot be used in the development machine

In this blog post, I will be focusing on Remote Debugging .NET applications through Visual Studio 2010. The screen shots correspond to a simple .NET application which would popup a MessageBox on a button click. Get the code here.

Configuring the Host Computer

The host computer is the computer containing the development environment. This is the system running Visual Studio 2010. To enable Remote Debugging, make sure the following ports are open on the host computer -

Ports	Protocol	Description
135	TCP	Required
500, 4500	UDP	Required if your domain policy requires network communication to be performed through IPSec

IPSec (Internet Protocol Security) is a protocol suite for securing IP communications authenticating and encrypting each IP packet of a communication session. Read more about IPSec here. However I am not going cover IPSec in this post.

You can check if the ports are open by using the telnet command at command prompt (telnet isn't installed by default on Windows 7. Install it from the "Turn Windows features on or off" section in the Control Panel).

Configuring the Remote Computer

To configure the remote system for debugging, either download and install the Microsoft Visual Studio 2010 Remote Debugger or copy the folder Remote Debugger from the Visual Studio install path, which is typically C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE. The Remote Debugger is available for three configurations - x86, x64 and ia64.

Visual Studio takes the .pdb files for debugging from the remote system for managed code. So copy the .pdb from the bin/Debug folder of the project to the remote system. Put this folder on share so that Visual Studio can access it later.

To enable Remote Debugging, make sure the following ports are open on the host computer -

Ports	Protocol	Description
135, 139, 445	TCP	Required
137, 138	UDP	Required
500, 4500	UDP	Required if your domain policy requires network communication to be performed through IPSec
80	TCP	Required for Web Server debugging

That's it; both the host and remote computers are up and ready for Remote Debugging.

Start the application on the remote system if it isn't running already. Also start the Visual Studio Remote Debugging Monitor on the remote system using the msvsmon.exe from the appropriate configuration folder in the Remote Debugger directory (The screenshot uses the x86 version).

Open the code in Visual Studio on the host system and select "Attach to Process". Typically Visual Studio identifies the .pdb file of the remote system if it can. If it doesn't, open the Modules screen from Debug -> Windows -> Modules. Check the Symbol Status of the .exe file which is being debugged and if it shows "Cannot find or open the PDB file" right-click on it and select "Load Symbols From" and specific the share path through the "Symbol Path".

Specify the breakpoints in the code and they would hit appropriately. There are two major bottlenecks in Remote Debugging -

• The code from which the executable was built should be available at the time of debugging
• The above mentioned ports should be open for debugging. Most of the developers wouldn't have administrative privileges on the client systems, so they would require people with sufficient privileges to open up these ports. However several organizations wouldn't prefer opening up ports for incoming connections

Before I conclude, here's an article from MSDN on how to Set Up Remote Debugging. Visit it if you would like a deeper insight into Remote Debugging.

TVPs Vs XML - An Analysis Through .NET

A common requirement of .NET applications that connect to databases like SQL Server is passing Lists and Arrays to the database server. Since most of these databases are relational, transferring these Lists of Objects isn't really straight-forward.

One way of passing these objects would be to send individual properties of these Objects to the database server, one object at a time. Obviously this is highly inefficient. Understanding this, various database management systems have provided various approaches to achieve this.

There are two ways of passing such content to SQL Server -

1. Generating an XML from the List and passing it as a parameter
2. Creating a Data Table from the List and passing it as a Table Valued Parameter

Passing Lists and Arrays as XML to SQL Server has been a common approach since SQL Server 2000. From SQL Server 2008, Microsoft introduced a new way of passing Lists and Arrays - Table Valued Parameters.

In this post, I will be evaluating the performance statistics of XML and TVP though a .NET application. I will be doing this through the RetrieveStatistics method of the SqlConnection class. This method retrieves the statistics of an operation through an IDictionary which contains various performance metrics.

This happens only when the StatisticsEnabled property is set to true before executing the query/stored procedure. These metrics can be reset using the ResetStatistics method of the SqlConnection class. Here's a typical usage of this method -

The dictionary contains several provider statistics. There are 18 values that can be obtained from the Microsoft SQL Server provider. A detailed analysis of the metrics that can be obtained from the dictionary is available here - http://msdn.microsoft.com/en-us/library/7h2ahss8(v=vs.80).aspx.

The .NET application that is being used in this post to compare the performance of TVP and XML captures these 18 metrics into an object. However the final analysis focuses on three of these properties - Execution Time, Bytes Sent and Bytes Received.

The application primarily inserts records into a table. The number of records and the value of the records are decided by the application. To understand the effect of load, the numbers of records are raised from 25 to 2500 in gradual steps of 25. For an accurate measurement each step is repeated for 25 times and an average of the metrics obtained is considered.

Here are the graphs obtained for Execution Time, Bytes Sent and Bytes Received -

Execution Time (Records on X-Axis and Time on Y-Axis)

Bytes Sent (Records on X-Axis and Bytes on Y-Axis)

Note - The Bytes Sent for OpenXML and Nodes are almost the same.

Sent Received (Records on X-Axis and Bytes on Y-Axis)

Note - The Bytes Received for TVP and Nodes are the same.

The observations that I could conclude are as follows -

• The execution time for TVP is less than XML (Using OpenXML took more time than Nodes). The execution time of OpenXML was far higher than TVP
• The bytes sent for XML was higher than for TVP. This is probably due to the XML tags that had to be added for the transfer
• The bytes received in all the three cases were constant irrespective to the bytes sent. Though the number of bytes received for TVP and Nodes was the same, the number of bytes for OpenXML was slightly higher. This might be due to the procedure calls of sp_xml_preparedocument and sp_xml_removedocument

On a whole, TVPs look more promising than XML. However this was a very preliminary test and the actual results might vary in live environments.

The SQL scripts and the Visual Studio project used in this post can be found here and the raw numbers are here.

Compiling Visual Basic, C# & F# Code Into A Single DLL Using msbuild

Recently I had a requirement in which a C# file had to be included in a VB.Net project. I knew I couldn't add the file directly to the project, so I cut over the C# file to VB.Net. At first this seemed pretty obvious since Visual Studio maps every project to a programming language and uses an appropriate compiler to compile the files in the project.

But giving it a deeper thought, I felt something didn't fit the picture. Every project, rather every file that we compile using the .NET Framework generates Common Intermediate Language which is used by the Common Language Runtime. It is because of this CIL that we have language interoperability in .NET. So theoretically speaking, code written in different languages should be able to exist in a single assembly.

Though I couldn't find a way to directly add a C# file into a VB.Net project, I found a way to create a DLL from compiled VB.Net and C# code through a concept of the .NET framework called Multi Module Assemblies.

In .NET, the minimum unit of deployment is an assembly; Dynamic Link Libraries (DLLs), Executables (.exe), etc. are all assemblies. An assembly can contain multiple files like resource files, the manifest, etc. It can contain files of another type called netmodules. A netmodule is a unit of compilation. It cannot be deployed directly but can be linked into an assembly. netmodules written in different .NET languages can be linked into a single assembly.

Unfortunately Visual Studio wasn't built for Multi Module Assemblies, hence there's no way to create a netmodule from a project in Visual Studio. However msbuild allows us to create a netmodule from the projects. There are typically three changes that have to be done to the .proj files (.fsproj, .vbproj, .csproj, etc.) for msbuild to generate a netmodule -

1. Changing the OutputType to Module. Visual Studio doesn't recognize this output type, so even when you change this in the .proj files, Visual Studio won't be able to show it in the Project Properties
2. Exclude the AssemblyInfo file from the project. netmodules aren't supposed to have an application manifest. If they do, the linker won't be able to resolve which application manifest should be used when we are building an assembly from multiple netmodules
3. If one netmodule is dependent on another netmodule, a reference to the dependent module should be specified using the AddModules XML node in the .proj file. Similar to DLLs, netmodules cannot be circularly referenced

Once the .proj file has been modified, the project can be built using msbuild. If the project is a part of a solution, building the solution will create a netmodule for this project - msbuild solution_path

Once the netmodules are created using msbuild, a DLL can be created from them using the Assembly Linker of the .NET framework - al /t:library /out:"path_to_dll" path_to_netmodules

With this we have a DLL which contains all the netmodules bundled into one. To understand this better, let's consider an example - I have three projects, one in F#, VB.Net and in C# under a solution called MultiLanguage.

The .fsproj, .vbproj and .csproj have to be changed as mentioned above. From the netmodules obtained from msbuild, the DLL can be created using the assembly linker.

The sample solution and the build script can be downloaded from here. I automated the .proj changes using Microsoft's Build Engine API. You can give it a try here; it takes the .sln file as an input and modifies all the projects in the solution to generated netmodules.

Windows Phone 7 - Hello World!

Ever since Nokia announced its partnership with Microsoft and made Windows Phone 7 its primary smartphone operating system, I anxiously wanted to give it a spin. So in this post, I am going to start by setting up a development environment for Windows Phone 7 followed by a simple Hello World example.

Windows Phone 7 is primarily a successor of the Windows Mobile Platform (the last one in the line being Windows Mobile 6.5). It's a major revamp in Microsoft's Mobile strategy in terms of its application development model.

Windows Phone 7 no longer supports unmanaged code (Win32 and C++ are gone). It is purely built on Managed Code covering Silverlight, XNA and the .NET Framework (Programming Languages include C# and VB.Net). Here's a nice article on Windows Phone 7 application development model - http://blogs.msdn.com/b/abhinaba/archive/2010/03/13/windows-phone-7-series-programming-model.aspx.

To get started with Windows Phone 7 development, download the Windows Phone Developer Tools RTW (Release To Web). It includes the following tools -

1. Visual Studio 2010 Express for Windows Phone
2. Windows Phone Emulator Resources
3. Silverlight 4 Tools for Visual Studio
4. XNA Game Studio 4.0
5. Microsoft Expression Blend for Windows Phone

If you are already using a higher version of Visual Studio or Expression Studio, this toolkit will install extensions to the existing IDEs. The release is also available as an .iso image. Though the Windows Phone 7 toolkit comes equipped with Expression Blend, for most of the basic applications the design mode of Visual Studio should be sufficient.

To get a hands-on experience with Windows Phone 7, let's take a simple application which accepts a username and greets him/her in the next page. The best part of Windows Phone 7 development is that it is pretty much in the lines of other Visual Studio project types we are familiar with like the Windows Forms Applications, WPF Applications and ASP.Net Applications where in the UI can be created easily using the designer and the business logic is maintained in a code beside event driven model in another .cs or .vb file depending on the programming language.

As already mentioned, Windows Phone 7 is built on Silverlight and thereby every screen has a corresponding XAML (eXtensible Markup Language) page, each having its own code file. Movement between these XAML pages is made possible through an instance of a NavigationService class for every page.

There are three important methods provided by NavigationService - Navigate, GoForward and GoBack. Navigate takes a Uri instance specifying the URI location and loads the XAML page specified.

NavigationService.Navigate(new Uri("/NamePage.xaml", UriKind.Relative));

GoBack and GoForward loads the previous and next pages respectively.

NavigationService.GoBack();
NavigationService..GoForward();

Parameters can be passed across XAML pages by appending them to the URL and retrieving them using the QueryString property of the NavigationContext object. This concept of Windows Phone 7 was very surprising. Since Mobile Applications are very similar to Windows Applications, passing parameters through URLs like Web Applications looked a bit off-track.

NavigationService.Navigate(new Uri("/NamePage.xaml?name=" + name, UriKind.Relative));
string name = NavigationContext.QueryString["name"];

Download the example code here.

Introducing ADO.NET Entity Framework

This is the third and final part of my ORM series in which I am going to introduce the ADO.NET Entity Framework, an in-built Object Relational Mapping model of the .NET Framework.

Similar to the previous post, this one also covers the same four principles -

• Configuring a .NET project for the ADO.NET Entity Framework
• Inserting data from Objects directly
• Retrieving data using Object Lists an LINQ
• Changing the Database Management System

Though the screen-shots and the example codes emphasize on C#, the principles are same for all the .NET languages.

Requirements to run the example in this article - Visual C# Express, SQL Server Express and MySQL. Download Visual C# Express here, SQL Server Express here and MySQL here.

Configuring a .NET project for the ADO.NET Entity Framework

To configure a .NET project to work with the ADO.NET Entity Framework, an Entity Data Model is added to the project. The Entity Data Model is primarily a schematic representation of the database tables stored as a XML file. Each of these tables is converted into a class and foreign key relationships between these tables are maintained as Lists inside the objects.

Visual Studio provides a Wizard to create Entity Data Models. Right click on the project and select New Item from the Add menu. Choose the ADO.NET Entity Data Model template from the chooser. There are two approaches available to build the Entity Data Model - Database First Approach and the Model First Approach.

In the Database First Approach, the Entity Data Model is generated from existing table structures. The wizard allows the developer to setup a connection by providing the Database Server name and the Database name. The wizard also allows developers to choose tables, views and stored procedure that are to be a part of the Entity Data Model.

In the Model First Approach, the Entity Data Model is created from scratch and the database tables are generated based on this model.

Once the wizard completes, a designer opens up which shows a schematic representation of the generated .edmx file. This file contains the table mappings as a XML along with a code behind file with a .Designder.cs extension for the auto-generated classes corresponding to the tables. However a drawback with the Entity Data Model is that it combines all the classes into a single file which makes manual maintainence a little difficult.

I used the Database First approach to create the sample application to insert and retrieve data. You can get the MySQL and SQL Server scripts along with the Visual Studio solution here.

Inserting data from Objects directly

The Entity Model generates a class which inherits from ObjectContext. This class acts as a data manager to connect to the Database Management System to retrieve, insert, update and delete records.

ADO.NET Entity Framework maintains the records of the table as a List of objects. To insert new records into the table, create new objects, add them to the appropriate lists and then save the changes using a ObjectContext instance. The following piece of code shows how objects can be persisted -

using (Context context = new Context())
{
// Create the object
context.Objects.Add(object);
context.SaveChanges();
}

ADO.NET Entity Framework does a wonderful job while storing objects with foreign key dependencies. These dependencies are maintained using lists as objects and while storing these records, the appropriate identity keys are inserted into the child tables. However this feature is limited to a few DBMS like SQL Server.

Retrieving data using LINQ and Object Lists

The ADO.NET Entity Framework retrieves data from the back-end tables in the form of object lists. So accessing records is as simple as iterating through these lists.

using (Context context = new Context())
{
foreach (Object object in context.Objects)
// use the object appropriately
}

Since data retrieval is in the form of lists, developers can piggy-back on an other .NET framework feature - LINQ (Language Integrated Query). LINQ makes conditional querying of data a lot easier.

using (Context context = new Context())
{
var objects = from object in context.Objects where "condition" select object;
// use the objects
}

Changing the Database Management System

Before changing the DBMS of the Entity Model, it is important to understand how the ADO.NET Entity Framework stores the connection strings and the mapping between classes and the back-end tables. The connection string is stored in the App.Config file of the project and the table mappings are stored as a XML in the form of the .edmx file as mentioned earlier.

Unfortunately the ADO.NET Entity Framework varies it's implementation with the DBMS. Because of this modifying the XML manually isn't easy. For Example, ADO.NET Entity Framework does not support foreign key constraints in the form of lists for DBMS like MySQL.

The sample application contains another Entity Model which connects to a MySQL Server containing similar tables. To use MySQL with the ADO.NET Entity Framework, an connector is needed. The MySQL Connector/NET is available here.