Logging .Net

This is my foray into building .net systems. My boss says, "Lets build our new enterprise system in pure .NET. Lets do it in 6 days." Obviously this upset me quite a bit. Rebuild our VB6 middleware in .NET in 6 days? That's going to be tough. "Don't forget to use best-practices" he added.

Ok, our enterprise middleware engine consists of auditing, messaging and transmission (Databases), error handling and id generation. Query best practices in .net and one usually encounters Enterprise Library Blocks. Great half my work is done. We'll implement auditing in stored procedures, database access through the Enterprise library blocks - which leaves error handling and id generation to be built. ID Generation is nothing but a business object which calls stored procedures so that's basic. So I spend most of the 6 days learning ASP.NET and trying to get logging to work on .NET using best practices.

Enterprise Library Logging Block

This guy comes with the Enterprise Library and its configured through the app.config file. The most major problem dealing with this guy is where to put the app.config. Here is what you need to know:
1. ASP.NET - if you use logging directly from ASP.NET you app.config file is also your web.config file. Put all your configuration entries in there. Beware there is a hierarchy. I haven't really tried it myself, but after reading about this bugger for 2 days, I picked up some things.

2. .NET application - this is the easiest. In your application solution, add a configuration file called app.config. When you compile, this file will be renamed to "appname.exe.config" - replacing appname with your own application name.

3. COM+ application - This bugger is not documented in the Enterprise Library docs, but I suppose one needs to know a bit about .NET Enterprise Services to know about this. The app.config file for COM+ applications is called application.config and its located in a specified folder in the COM+ application property sheet. In COM+ Services MMC, go to the Activation Tab and set the application directory. Drop in the application.manifest and application.config file in that folder. It should work - but it hasn't for me. That's why I abandoned Enterprise Library logging and moved on to Log4Net.

So since I couldn't get Enterprise Library logging to work with COM+ applications, I tried Log4Net by Apache. If you want to persist with Enterprise Library Logging, you should know the following:

1. Use the Enterprise Library Configuration program to edit your app.config or web.config or application.config files.
2. In Enterprise Library Logging Block there are categories, formatters, and trace listeners. The default trace listener adds entries to the system event log. Its called Formatted Event Log Trace Listener. This is fine. But you want to add your own trace listener such as FlatFile Trace Listener. Not only that!
3. You need to Set the category for the TraceListener to listen to. Default category is General. You can add categories. Categories is also what you specify when you create your LogEvent object in code. If you try to add a LogEvent with a category which is not defined in your app.config file, you won't be able to see the event in your logs. This might cause you confusion when you try to run the sample code in the Enterprise Library help system against a default app.config file.
4. You should also set your TraceListener formatter and filename (if its a flat file trace listener).

Ok let's move on to log4net.

Log4NET

Using log4net, the documentation says you can programmatically set the parameters without the need of an XML configuration file. That's nice if there was sample code in the documentation to show you how to do this. The whole problem with Enterprise Library Logging Block was not being able to verify WHICH app.config file it was using - if it was using one at all. Presumably, if it used one, it would log properly. But seeing as the Logging object encapsulated the Enterprise Library Logging Block functionality to provide an additional layer of abstraction, and we considered whether this Logging object should be COM+, a .NET private assembly or a .NET shared assembly, figuring out where exactly the app.config file should be is no easy task. Luckily in log4net you can specify where to put the log4net.xml config file - except that now this location had to be hardcoded which is not good practice anyway. The code for that is:

XmlConfigurator.Configure(New System.IO.FileInfo("c:\log4net.xml"))

I found out, you can solve the hardcoding of the location by using the app.config settings which bring us back to the original Enterprise Library Logging Block problem.

To write to the log:

mlog As ILog = LogManager.GetLogger("Module")
mlog.Info("Message")

Most examples use LogManager.GetLogger(GetType(object)) but I don't think it matters. Also there are several predefined log types (INFO, DEBUG, ERROR and FATAL) so you have to use mlog.debug("Message") etc. The logs types are hierarchical so log level DEBUG - presumable is the lowest level meaning everything will be logged.

Comparisons

Technically, I don't think log4net is as flexible as Enterprise Library Logging Block but I'm forced to use it because I can't get Enterprise Library Logging Block to work for me. For my project I encapsulated both logging libraries in my Logging module and use the app.config settings to turn things on and off.

In case you're wondering, you can get values out of your app.config settings in the following way:

Dim strValue as String = System.Configuration.ConfigurationManager.AppSettings("MySetting")

Token knowledge

Standards are great. But lack of documentation is bad. Documentation needs to exist at various levels. You should have very technical documentation, for people interested in all the knitty gritty stuff. You should also have superficial overviews for people who aren't interested in the details. And then you should have documentation like this. Enough info for you to follow and do something, but not too much that your head spins. I had to search around for half a day to pick up enough technical information to make it possible. Then with very basic understanding of how things 'should' work, I put together the system.

The thing is, with security applications, a little knowledge is a very bad thing. If you design a security system without a proper understanding of the entire architecture, you may just be creating a very hackable system, will multiple holes just waiting to be exploited. So I don't claim that this system is unhackable or even secure. So since you've read this far without knowing what I'm talking about - here it is. The little project intends to utilize your PKCS11 compliant security token to extract a pin-locked secret for whatever you decide to do with it.

The idea is that a security token is a hardware device which stores some sensitive information, sort of like a physical key contains ridges and valleys. Let's go thru some terminology to get you started when reading other docs about PKCS11.

PKCS11 is the standard API for hardware security tokens. This can be smart cards, USB smart tokens, or Hardware Security modules. All these devices have one thing in common. They store keys. Some also store certificates. That's another PKCS standard - I think its PKCS12 but you have to check that.

Token refers to the device. To access your token you need a PIN. PIN can mean text as well - not just numbers. Inside the token there is a keystore. The keystore stores keys. Think of it like a key chain. To access your keystore you may need a password. Keys are stored in a way that you can easily access a key by an alias - or a name.

So our basic process is thus - Access the key, then unlock it, open the keystore and get the key we want based on its alias. Simple? Yes. By the way, the language we will use is Java - so if you're looking for a way to do this using another language - look elsewhere.

Setup - First install your token drivers. Make sure your OS can recognize the token and that the token has PKCS11 interfaces. There will be a library which provides this interface. On Windows this will be a DLL, on Linux, a .so file. You will need to find this file, or guess what it is, if there's no documentation to help you. For Safenet tokens and smartcards its dkck201.dll. Once you found this file, you need to create a config file - which is a text file. It's possible to go without the text file but lets just make the file.
Here is what the file contains (pkcs11.cfg).

name = SmartCard
library = c:\windows\system32\dkck201.dll
slot=14

Friendly name of SmartCard, location of library and slot. Oh yea... another definition. Slot refers to the the smart card reader, in case you plugged in several readers. You can find out the slot from your smart card reader driver, or you can iterate all the slots until you find the card you want. I won't bother with this.

Once you have your config file, its time to code

String pkcs11ConfigFile = "d:\\pkcs11.cfg";
Provider pkcs11Provider =
new sun.security.pkcs11.SunPKCS11(pkcs11ConfigFile);
Security.addProvider(pkcs11Provider);

Here you tell the Java Security system to configure itself based on the config file. You're adding a provider - which is your smartcard, token or HSM.

Work - Now you do your stuff. A little guide to useful string conversion functions. Call toCharArray() to convert a String to a char[]. Call getBytes() to convert a String to a byte[]. Call new String(byte[]) to convert byte[] into a string.

            KeyStore ks = KeyStore.getInstance("PKCS11");
            ks.load(null, defaultPIN.toCharArray() );

Here we open our token and unlock they keystore using the default pin. Note the keystore isn't pin protected.

            SecretKeySpec key = new SecretKeySpec(secret.getBytes(),"DES");

Here we're creating a secret key. Note that secret is a string and we're creating a SecretKey object from our string. I used DES mainly because it worked for our 8 char secret. Your results may vary. If you're using a specific encryption algorithm, you should probably create a key for that algorithm. I'm not going to show you how here.

            ks.deleteEntry(keyalias);
            ks.setKeyEntry(keyalias,key,sPIN.toCharArray(),null);

We delete any keys at our keyalias, then store our new key in the keystore, using the keyalias. That's it!

But wait. How do you get the key out, after you've put it in? Here's how

            KeyStore.SecretKeyEntry skEntry = (KeyStore.SecretKeyEntry) 
                ks.getEntry(keyalias, null);
            byte data[] = skEntry.getSecretKey().getEncoded();
            String secret = new String(data);

Get the key store entry under the keyalias. Get its encoded value - this will not work for all key types - but it works for this one. Then convert the byte array to a string. There's your secret. Now you can do what you want to it. Notice you have to unlock and load the keystore before you do this - so the PIN is necessary. And we're done. A quick 5 minute project for learning enough about PKCS11 to impress other people who don't know anything about PKCS11. Impressed? Thankful? You're welcome.

JDBC Connection pools in Sun Application Server

Here is a little problem which lasted 2 hours. The question is, how does one use JDBC Connection pools managed by the Sun Application Server? If you look in the administration guide, it won't tell you anything. Ask a bunch of experts on Sun forums, and they tell you nothing. This little entry should resolve your problems.

There are two main things to consider when setting up the connection pools.
1. JDBC Driver - Your DB of choice comes with a JDBC driver. Your Sun application server does not. So your first order of business is to add your JDBC driver to the Sun Application Server classpath. The surest way to do this is by going through the web admin console and adding the full path to the JDBC driver JAR file in one of the classpath boxes. Go to Application Server -> JVM Settings -> Path settings. I added the MySQL JDBC driver to Classpath suffix. Now you can set up connection pools and JDBC resource. This is the straight-forward stuff which IS documented. Then when you ping the connection pool, you should get Ping Success instead of class not found.

2. JNDI Name - If you use Netbeans, It might set up the JNDI name incorrectly which is the cause of the 2 hour search for answers. Netbeans smartly uses "java:comp/env/<JDBC Resource Name>" when what really worked was simply "<JDBC Resource Name>". Thus the code to get the datasource should be

javax.naming.Context c = new javax.naming.InitialContext();
return (javax.sql.DataSource) c.lookup("<JDBC Resource Name>");

After all that, don't forget to restart the Application Server. Now you can thank me for saving you 2 hours.

File upload

Today I was racking my brain thinking about how to transfer some sensitive files from one secure server to another. The secure server had most of its ports blocked, so FTP or other file transfer methods were out. I didn't want to bother with a cumbersome HTTP Proxy. Besides the files were secret so it shouldn't go to a third party. I wish there was a module I could enable on my web server to a HTTP file upload. This is it. It's two PHP page and an upload folder. I snipped it off a website, took out the file type and file size restrictions and I have a PHP module that will accept any file. This is potentially bad, security wise but we can practice security by obfuscation - meaning as soon as the file transfer is done, delete the module.

Forgot your Windows password?

This happened to me, not on my laptop but on a seldom used development box. Rather than reinstall everything again, I remembered the password. Yay! But what if I can't remember. Try this:

http://home.eunet.no/pnordahl/ntpasswd/

Microsoft Message Queue from Classical ASP

After days and days of doing absolutely nothing, I needed something to reaffirm my skills. Here it is - a task assigned to a couple of junior programmers. They weren't able to complete. Their supervisors weren't able to help them. Enter technosock! I took on the challenge and succeeded. Yay! The problem: We have a long running ASP transaction. We want to use MSMQ to store the event and do the processing later. Task: Use ASP to store the event.

Setup

MSMQ needs to be installed on a domain controller. I think so. So create a Windows 2003 server, and a new domain. I make it the Primary Domain Controller. I also install the Microsoft Message Queue components. All of them.

On the client side, I tried writing VB6 code from my PC - it didn't work. My guess is that you have to be a domain member. Fuck that! I'll write ASP and host it directly of the server in question. So I install IIS and enable Classical ASP (its disabled by default).

Code

There are four functions which I tried. First of all, we need to create the queue. Secondly, we write a message to the queue. Thirdly we read from the queue and fourth we delete the queue. Straight forward? Yes.

Create the queue


Set qi = Server.CreateObject("MSMQ.MSMQQueueInfo")
qi.PathName = ".\MyQueue"
qi.Create


qi is a queue info. MyQueue is the name of the queue. .\ means use the local hostname.

Send a message the queue


DIM MQ_SEND_ACCESS, MQ_DENY_NONE
MQ_SEND_ACCESS = 2
MQ_DENY_NONE = 0

Set qi = Server.CreateObject("MSMQ.MSMQQueueInfo")
Set msg= Server.CreateObject("MSMQ.MSMQMessage")
qi.PathName = ".\MyQueue"
Set objQ = qi.Open( MQ_SEND_ACCESS, MQ_DENY_NONE )

msg.Label = "Label"
msg.Body = "Body"

msg.Send objQ, 0


objQ is the queue. Use qi.Open to get it. The parameters MQ_SEND_ACCESS and MQ_DENY_NONE are constants. The values are 2 and 0. Create a message object msg then send it to the queue objQ.


Receive a message from the queue


Dim qi
Dim ReceiveQ
Dim msg
DIM MQ_DENY_NONE
Dim MQ_RECEIVE_ACCESS

MQ_DENY_NONE = 0
MQ_RECEIVE_ACCESS = 1

Set qi = Server.CreateObject("MSMQ.MSMQQueueInfo")
Set msg = Server.CreateObject("MSMQ.MSMQMessage")

qi.PathName = ".\MyQueue"

Set ReceiveQ = qi.Open(MQ_RECEIVE_ACCESS, MQ_DENY_NONE)

Set msg = ReceiveQ.Receive(3, false, true, 1000, false)
If msg is Nothing Then
     Response.write "No message was received"
Else
     Response.write "Message received<br>"
     Response.write msg.Body
End If

ReceiveQ.Close


ReceiveQ is the queue. Its opened with MQ_RECEIVE_ACCESS. Call ReceiveQ.Receive to get the message. The parameters 3, false, true, 1000, false are mostly flags. The value 1000 is the timeout in milliseconds.

Delete the queue


Set qi = Server.CreateObject("MSMQ.MSMQQueueInfo")
qi.PathName = ".\MyQueue"
qi.Delete


Same as the first snippet. Just change Create to Delete

Conclusion

There you have it. MSMQ in under five minutes. Ok so it took a couple hours to set up Windows Server 2003 and IIS and MSMQ. But after you start writing ASP code, its just a few lines of code. There's lots more you can do with message queues like sending binary objects, restricting who has access to the queue and so forth. Message Queues are a good idea. MS has some good tools but documentation is a but lacking in my opinion. I had trouble searching for most of the things - could be because my connection is terrible. Anyway I've done all the hard stuff so you don't have to.

Project Guest Manager

Sometimes when I'm bored or when there is a need, I need applications for my own use, or for fun. I'll highlight them here from time to time. This application I started one afternoon when I was bored. It's a guest list manager which is comprised of a read-only Pocket PC mobile client and a desktop application which allows you to enter guest names, number of guests, their category and which table they will sit. It's very Chinese banquet oriented because the tables seat a recommended 10 people.

I wrote the Pocket PC mobile client first. It took about 10 hours over friday-sunday. The desktop application also took about 10 hours, over monday-tuesday. The database is a pocket access database but as I found out, ADOCE has been discontinued for windows mobile 5 which makes my app obsolete before even being released. Thus my dreams of charging people for my 20 hours of work has gone down the drain. I hate you Microsoft. Pocket Access wasn't great but at least it filled a need. The need to have a database sync to pocket pc automatically. I didn't need the fancy table joins and macros. I didn't need super high performance.

Anyway the apps are simple. On pocket PC you need three files.

1. GuestManager.exe (which is an ARM based pocket PC app)


2. \My documents\GuestList.cdb (which is a pocket access database - converted from a desktop MDB)


3. \My documents\map.png (which is the table layout of the banquet you're organizing).

The trick is converting the MDB to CDB. You need an old Active Sync. If your Pocket PC can't work with an old Active Sync, you're out of luck.

On the desktop you need the GuestManagerDesktop.exe and the GuestList.mdb in the same directory.

The desktop app does some cool things.

1. It stores state information in an xml file. It remembers the location of the last database and reloads it.


2. We print 3 types of documents (alphabetical guest list, guests by table, guest by category) without use of special report components. It's not easy because it involves complex looping and special index and counters.

Some features of the desktop app.

• Add any number of tables and categories just by adding a guest to the table or category. The table and category are combo-boxes which allow test entry. It supports special characters (as far as i know) - Not sure about unicode though.


• Guest list is browsable by table and the application indicates unfilled, filled, and overfilled tables using colour cues.


• Able to delete and move guests by right-clicking on the guest name or selecting from the menu.


• Able to change guest details by selecting the guest, changing the details and clicking on Save.


• Save will either insert a new guest or update an existing one, depending on whether the guest already exists.


• Performs analysis of guest to table distribution.


• Console log allows you to revisit all the actions performed in that session.


• Allows printing of three kind of reports with checklist boxes:

a. table listing
b. alphabetical guest listing
c. listing by category

I'm pretty happy with the app. It could use a bit more polish such a splash screen or a nag window and some crippleware. Maybe the log could be persistant.