Monthly Archives: May 2007

New Milestone: My First Linux Application

TuxLast night I finished my first program that runs on Linux. If you haven’t guessed yet, its built on the PostgreSQL and Python application stack. The program is a server that checks and updates different files, while at the same time, has the ability to accept remote connections through telenet for administering the server.

I can’t go into deeper details about what the program does because of an agreement with my employers, however, I can talk about what I had to learn in order to put everything together. Later on in the week, I plan on posting a write up on Python and the Twisted Framework. While researching Twisted, it was maddening at the sparse amount of example code for some parts of the framework.  For me, it was authentication and integration of application code outside of protocols. Even now after completing everything, I’m still clueless on a whole slew of issues. I know most of the nerds out there will just say, ‘So? why don’t you just read through the source? It’s on your computer in plain sight.’ Since I’m still new to Python, reading through the source for the frameworks is like reading Chinese. As I’ve mentioned before, it all looks like a hodgepodge of pseudo code.

Windows Client for Amazon S3

I haven’t had time all week to work on the Windows Client for Amazon S3, however, I did promise to release the interface and the source code. The following links will get you either the installer or the source. This has only been tested on Windows XP and requires the .NET 2.0 Framework in order to run.

If you find any bugs or have any feature requests, please contact me.

Still Switching to Python: Iterating through dictionary lists

Last night while developing a module, I ran into a situation I never thought I would encounter, a syntax problem. Why the surprise? Python is fairly sparse when it comes to syntax. The lack of keywords, brackets, and semicolons makes code look more like a hodgepodge of pseudo code than a programming language.

While trying to implement a simple iteration through a dictionary list, I ran into a syntax error with the following code:

for k,v in dictList.items():
print "%s=%s \n" % k,v

This resulted in the following error: ‘ValueError: need more than 1 value to unpack’.

After a bit of searching and experimenting, I figured out that I was missing the parenthesis around the ‘extraction values’. This was confusing because most of the examples I found did not cover the more familiar  iteration patterns in C like languages. To correct the error above, you would type:

for (k,v) in dictList.items():
print "%s=%s \n" % (k,v)

During my search for the solution. Most of the examples I found for iterating through lists looked more like the following:

print "\n".join(["%s=%s" % (k,v) for k,v in dictList.items()])

 
The above accomplishes the same thing, just in a slightly more succinct format. To most programmers that come from a C background, this looks downright odd – kind of like comparing English to Old English. Looks similar but not.

Switching to Python

Yesterday I looked through a bunch of things related to developing in Python and PostgreSQL. One of the things I was researching was unit testing my code. Unit testing is the testing of individual software components. Many frameworks exist for doing this. These include NUnit, JUnit, MbUnit, and MS Unit Testing. Yesterday I looked at PyUnit, a port of JUnit to Python,  as a potential candidate for testing not only code for future Python programs, but also unit testing functions in PostgreSQL. For now, I think this will be sufficient.

As I’ve started to use Python more, I’ve yearned for a better IDE than Eclipse and PyDev. So far the only commercial IDE’s I have read about are WingIDE and ActiveState’s Komodo. I plan on looking at these sometime soon.

New Article

PostgreSQL: Notes about Quotes and Identifiers

Last night I was playing around with schemas and found some oddities with quotes. In version 8.0+, you can use ‘$$’ symbols to replace quotes. This gives the programmer some flexibility in terms of having to escape quotes within the text body. For example:

CREATE OR REPLACE FUNCTION NoticeNoEscaping() RETURNS VOID AS
$$
BEGIN
Raise Notice "See, no escaping required";
END;
$$
LANGUAGE 'plpgsql';

When creating tables, I noticed that defining a column name with quotes and without quotes results in two different column names.

Create Table QuotedColumn
(
“Id” Serial,
Primary Key(“Id”)
);

The following would throw an error because of missing quotes.

Create Table QuotedColumn
(
“Id” Serial,
Primary Key(Id)
);

I suggest dropping the quotes all together. To correct the error above, you would type:

Create Table UnQuotedColumn
(
Id Serial,
Primary Key(Id)
);

I also encountered a similar problem while implementing a schema. For this example, a table is being added to the schema, ‘test_schema’.

Create Schema “test_schema”;

Create Table “test_schema”.Customers
(
Id Serial,
Primary Key(Id)
);

When creating schemas using PgAdmin, quotes are automatically insert around the schema name. This results in having to use quotes to define the schema prefix each time a table within the schema is referenced. Try the following instead.

Create Schema test_schema;

Set search_path To test_schema;

Create Table Customers
(
Id Serial,
Primary Key(Id)
);

 In the above snippet, the schema,  ‘test_schema’, can now be referenced without quotes. Not only that, but I also set the schema for all code to automatically reference ‘test_schema’ by using the command ‘Set search_path To <schema name>;’.

If you would like more information on the topics discussed, please consult the following links:

My Weekend in a Nutshell: Seeing Downtown, Amazon S3 + .NET, and More Fun on PostgreSQL

Downtown Dallas Last week was pretty dull. As you can tell from my posts, its been dominated with the migration over to Linux. Eventually, I want to port this blog over to an open source platform. I’ve looked at a few and some of them have been fairly impressive. WordPress, Snurf, and b2Evolution are some of the candidates I’ve been considering.

On Saturday, I decided to visit some friends in downtown. I ended up at a bar called Petrus. You can check out what I thought about the place here.

Sunday I decided to take a break off of O.S.S. programming and look at some good old fashion .NET programming. The problem I was contending with that day was a lack of a Windows Client for Amazon’s S3 Storage Service. After doing a quick scan through Google, Codeplex, and Sourceforge, I did not find any free programs for interacting with this service. In response, I decided to whip up an interface. Admittedly, the interface is in rough shape right now. I just wanted to get something up quick and working. Before I release it into the wild, I’m going to clean up the interface a little bit. By clean, I mean move a few labels around. (what’d you expect? Its free.) This will be released by the end of the week – source code and the installer.

On a different note, today I had to learn about arrays in PostgreSQL. I needed to figure out how to do batch updates without the use of delimited strings. An article has been posted about this topic.

New Articles

*Update* 5/29/07

Source and Installer for the Windows Client has been uploaded. Please refer to the links below:

PostgreSQL: PL/PGSQL, Arrays, and Batches

When developing in T-SQL on SQL Server, I sometimes need to update or process multiple rows at once. This involved either multiple round trip calls to the database or sending a delimited string that would eventually be parsed into a matrix and then inserted into a table. Once you figured out the process for such operations, implementing a solution was quick.

In PostgreSQL, the need for such operations isn’t as dire because of the array data type. Instead of parsing and formatting data into a delimited string, arrays can be passed directly to a function (stored procedure). The prime benefit is less code to maintain and standardized functionality. In version 8.2, only basic data types are supported. Composite, user defined, and domain data types are not supported by arrays.

In order to create an array simply type:

Select ARRAY['one', 'two', 'three'];

In order to create a multidimensional array, type:

Select ARRAY[['a','one'], ['b','two'], ['c', 'three']];

Keep in mind when creating arrays, all data types in each element must be the same. Mixing data types in the same array will result in an error.

Now that we know how to create arrays. Let’s create a function to process these arrays. Consult the following example:

CREATE OR REPLACE FUNCTION MultiArrayRead(stuff varchar[][]) RETURNS VOID AS
$$
BEGIN
FOR i in array_lower(stuff, 1) .. array_upper(stuff, 1) LOOP

Raise Notice '%, %', stuff[i][1]::varchar, stuff[i][2]::varchar;

END LOOP;
END;
$$
LANGUAGE 'plpgsql';

If you’ve programmed in any language, the keywords and the structure of this code snippet should be fairly self-explanatory.

The first line can be broken down into the following:

‘CREATE OR REPLACE FUNCTION’ – In T-SQL, you had to define a statement that checked if a stored procedure existed and then called a command to drop the function. In pgsql, this one snippet takes care of detecting and dropping a function (stored procedure) of the same name or signature.

‘MultiArrayRead(stuff varchar[][])’ – ‘MultiArrayRead’ is the function name. The parameters accepted are defined between the parenthesis. In this case a two dimensional matrix of varchar elements called ‘stuff’. Please note that a varchar without a length defined as opposed to a varchar data type with a length( such as ‘varchar(30)’), will accept any length of string… well not any …. up to 1 GB, I think.

‘RETURNS VOID AS’ – Defines the return data type. This function returns nothing.

The ‘$$’ symbols replace the traditional single or double quotes. I’m using these symbols because it frees the developer from having to escape quotes more often within the definition of the function.

The fourth line of the snippet defines the beginning of the For loop. ‘array_lower(array_variable, matrix_column)’ and ‘array_upper(array_variable, matrix_column)’ return the first index  and the last index of the array. If your matrix has more then one column, you can return different bounds for a matrix’s columns by incrementing the second parameter.

I mentioned Batch commands in the title of the article. After passing a matrix and any other variables needed to modify  or retrieve information, iterate through the commands you would like to execute in the loop.

For Example:

CREATE OR REPLACE FUNCTION MultiArrayWrite(stuff varchar[][]) RETURNS VOID AS
$$
BEGIN
FOR i in array_lower(stuff, 1) .. array_upper(stuff, 1) LOOP

Insert into example_table(id, description) values
(stuff[i][1]::int, stuff[i][2]:varchar);

END LOOP;
END;
$$
LANGUAGE 'plpgsql';

The ‘::data_type’ (like in ‘stuff[i][1]::int’ ) appended to each element is used for casting the element to the data type defined in the columns of ‘example_table’.

Calling this function can be done by executing the following:

Select MultiArrayWrite(ARRAY[['1','one'], ['2','two'], ['3', 'three']]);

This is just a brief overview of arrays and batch executions. If you would like more information on these topics, please consult the following links.

Review: Petrus Lounge

Petreus Entrance
Petreus Patio
Petreus Lounge
Petreus Patio

Petrus Lounge

1217 Main St.
Dallas, TX 75202

Dress Upscale. ‘Schwanky’. Men were wearing nice shirts and pants with dress shoes. Ladies were dressed in skirts, cocktail dresses, nice top / jeans.
Atmosphere More of a place to go out with friends and relax. First floor has a bar and couches. Rooftop has a well lit patio and a view of the downtown skyline.
Crowd Very trendy and fairly mixed. I didn’t see any ghetto thugging that night. The average age was early-20s to early 30s. Sex ratio was about 60% guys, 40% girls.
Parking Valet at the front. Parking lot about half a block down the street is five dollars.
Prices $8-10 per drink. No cover charge.

Petrus Lounge is located in downtown off of Main St., about 3 blocks from Nieman Marcus or right by that gigantic high rise with green neon lights running down the sides. The lounge takes up the 4th floor and 5th floors (roof). If you have been to Fusion before, this club will remind you of some similarities – with the exception that it feels more crowded due to the smaller space. As I was walking in, I was greeted by some gorgeous women. This seemed to be the reoccurring theme the entire night.  After meeting some friends, we decided to move from the VIP booths in the lounge area to a table on the patio. Overall, the table on the patio level is a lot nicer than the booth. When you walk out, a view of the downtown skyline and a nicely lit atmosphere greets you. Compared to the lower level, it feels a lot more open.

The Good

Aside from the superb atmosphere and view, the service was fairly good for a Saturday night. Drinks weren’t watered down and the bar tenders were pretty quick for a crowded night.

The Bad

I know clubs and bars can get crowded quickly on Saturday nights. Petrus Lounge was no exception. At around midnight, the patio area was packed. Strangely though, the lounge area was pretty much half-empty. Turns out everyone just got drinks in the lounge area and then went upstairs. Aside from the crowded conditions, the music could use some improvement and the dance area, while cool because it sits on top of a fountain area, could use more space.

Overall

If you’re looking for a place to spend time with friends for a relaxing night and maybe meet new ones, I highly recommend it. However, if you’re looking for a dance club, a place to blow off some steam, or go on a bender, I recommend you go somewhere else. At least the night I went, the crowd was low key.

How I Got Interested in Computers

I’ve played my fair share of console and computer games. Sometimes I wonder if I’ve played way too much and I think to myself. “Gee whiz , would I be a different person today if I was never exposed to this?”

 In short, I think I would be a very different person. One of the primary reasons I became interested in computers and eventually information technology in general was because of video games. In its early hey day, getting a PC game to run on an old 486 IBM Clone was a challenge. The primary limit to getting a game to run was memory, or ‘conventional memory’. It would take hours to get a game working. The more cool stuff in a game – the more hours spent squeezing every single available byte of memory out of the computer.

 If you needed assistance, finding it was hard because technical knowledge was even scarcer than it is today. At that time, the internet was confined to universities and major telecommunication companies, so no help forums. The only public alternative was the local Bulletin Board Systems, or public servers that you contacted with a dial-up modem where people could chat, share files, and play games, which had maybe 10 users online at a time.

 As a side effect of really wanting to play PC games, I had to learn how to configure and setup complicated, memory hogging, and sometimes buggy programs. This eventually got me exposed to the earliest forms of networking technology with modems and dial-up servers. If it weren’t for great PC games, I would’ve never taken an interest in computers.

Developing on an Open Source Platform versus the Microsoft .NET Platform

It’s been 11 days since I switched my development environment over to Linux. I’m still running into new problems and figuring out new solutions on a daily basis. My latest problem concerned running a simple bash shell script.

As for news on software development, I’m still looking over PostgreSQL and will probably post another article on it in a few days. I’m about to start heavy development on that end. I’m also looking into Python. There are two major frameworks I’ve been researching and developing with –Twisted Framework and Django. I’ve had some time to read through a lot of documentation concerning both and so far things look promising. I encourage anyone out there to check them out.

As a Microsoft developer, how does developing on an open source platform compare to the Microsoft platform? Before I get into that, I need to explain the differences between the two development philosophies behind these platforms. First of all, the philosophies on developing on the two platforms are very different. Microsoft makes superior tools to anything I’ve seen on a free platform, but I think that stems from their development philosophy – use great  tools to expedite and ship code faster. I think the need for these tools has stemmed from the design of their application-tier (.NET) languages because they are extremely verbose in nature. This has resulted in the development and need for effective code generators, snippets, macros, and auto completion tools found in the Visual Studio IDE.

Because I haven’t been developing on an open source platform for long, I can’t comment in depth or with too much knowledge yet. From what I have seen so far with the design of different programs, things are taken from a minimalist approach.

The open source application PostgreSQL is built to serialize and query data. Their are no extra bells and whistles – such as Reporting Services, Notification Services, Data Warehousing, Message Queues, etc. that are found in SQL Server 2005. This doesn’t mean PostgreSQL is inferior. Instead, the application fulfills its primary design goals extremely well. Because of its minimalist approach, it can be quickly deployed and configured quickly. It took me less than one week to get myself up to speed on administration, developing, and deploying on that platform. 

Same thing with Python, although their seems to be just as many extensions and frameworks out there as the .NET platform. The beauty of Python is that it takes less keystrokes to accomplish the same tasks in C# or VB.NET. This saves not only time but diminishes the need for all those elaborate code generation tools found in Visual Studio 2005. Although, sometimes I find myself wishing for Intellisense. Either way, both philosophies can build software quickly. It’s just one alternative is a lot cheaper than the other.

New Article