Tuesday, February 22, 2011
After attempting to import my equity trades into Intuit's TurboTax 2010 Home and Business edition, I discovered that support for importing CSV files has been removed. So, I decided to roll my own program.
You can download the source
here or download the Windows executable
here. Please note, the .NET 4.0 framework is required. After executing the program, you need to enter in the full file path [directory and file name] in the file text box and the full path for the directory where you want the file to be created. A 'TXF' file should be created afterwards in the specified location.
You can import the TXF file in TurboTax 2010 by opening your current tax return and selecting File -> Import -> From Accounting Software. Check the details before finishing the import to make sure everything looks right.
If you want more details about the TXF file format or the CSV reader I used in the program. Please check out the following links.
TXF Format
LumenWorks CSV Reader
I've only tested the TXF files from this program on Turbo Tax 2010 Business and Home edition. This software is provided without warranty or support.
Wednesday, October 28, 2009
One of the sites I maintain was unable to serve pages due to the server timing out. At first, I suspected it was a bandwidth issue, since traffic was significantly elevated due to a recent ad campaign that was initiated. On close examination, I noticed the system's resources were barely taxed at all. Hard disk I/0, CPU usage, bandwidth, and memory were either normal or barely used.
I then remembered a similar incident with another site that showed the same symptoms, except the site was hosted on an Apache server. The problem was that Apache was consuming too much memory and was unable to fulfill requests fast enough due to having a high connection timeout setting. The fix was to limit the amount of processes spawned and reduce the connection timeout setting.
Since memory and CPU usage was not an issue, the only other suspect was to check the connection timeout setting. The Connection Timeout setting is located under:
-
Open IIS Manager
-
Expand websites
-
Select target website's properties.
-
Select Website tab
On default, it is set to 120 or 900 seconds – a very long time, especially if a web server has a queue of 1000+ objects to serve. My recommendation is to set the Connection Timeout setting to 2-5 seconds. This will free up the amount of connections available to users immensely. In addition, there are other optimizations to threading and compression that aren't enabled by default.
The following tweaks will help manage processes and threads for serving pages. In the %WINDIR%\Microsoft.NET\Framework\v1.1.4322\CONFIG\machine.config, set the following:
Setting
|
Default |
"Optimized"
|
| maxconnection |
2 |
12 * #CPUs |
| maxIoThreads |
20 |
100 |
| maxWorkerThreads |
20 |
100 |
| minFreeThreads |
8 |
88 * #CPUs |
| minLocalRequestFreeThreads |
4 |
76 * #CPUs |
Enabling file compression can free up a significant amount of bandwidth and increase the number of objects served at a time. First, give the folder where the compressed files will be stored write permissions to the IIS User. By default, this is “%windir%\IIS Temporary Compressed Files”. The IUSR_{machinename} will need the write permission to the folder.
Next, you need to enable Compression in IIS.
-
Open IIS Manager
-
Select the websites folder
-
Right click and select properties.
-
Select the Service's tab
-
Enable Compress application files
-
Enable Compress static files
-
Set max size to something sensible.
Then enable the direct Metabase editing. This allows you to edit the metabase while IIS is running.
-
Open IIS Manager
-
Right click the computer icon
-
select properties
-
Check the checkbox 'Enable Direct Metabase Edit'
Afterwards, you will need to open the IIS metabase by editing the file, %WINDIR%\system32\inetsrv\metabase.xml. Then find the attribute HcDynamicCompressionLevel and change the setting from 0 to 9, zero is the default setting. This will compress the pages as much possible without straining the CPU too much. There will two HcDynamicCompressionLevel attributes, each one under IIsCompressionScheme element that set deflate and gzip behaviors. Save the file.
Next entails adding the file extensions for static and dynamic pages that will be compressed. To do this, open the command prompt and open the directory C:\InetPub\AdminScripts\. Then execute the following to enable compression for the most commonly served files.
cscript.exe adsutil.vbs set W3Svc/Filters/Compression/GZIP/HcFileExtensions "htm" "html" "txt" "ppt" "xls" "xml" "pdf" "xslt" "doc" "xsl" "htc" "js" "css"
cscript.exe adsutil.vbs set W3Svc/Filters/Compression/DEFLATE/HcFileExtensions "htm" "html" "txt" "ppt" "xls" "xml" "pdf" "xslt" "doc" "xsl" "htc" "js" "css"
cscript.exe adsutil.vbs set W3Svc/Filters/Compression/GZIP/HcScriptFileExtensions "asp" "dll" "exe" "aspx" "asmx" "ashx"
cscript.exe adsutil.vbs set W3Svc/Filters/Compression/DEFLATE/HcScriptFileExtensions "asp" "dll" "exe" "aspx" "asmx"
IISreset.exe /restart
Do not compress images, as they are already compressed [jpg, gif, tiff, etc.]
Adjusting the IIS Application Pool can help resource utilization. The settings I changed were:
- 'Limit Request Queues' setting under the Performance tab. Depending on how many CPUs, Worker Processes, and memory available, this setting can increase performance. The machine I was tweaking this setting on had 4 CPUs and 8 GB of RAM. I increased the limit from 4000 to 10,000 with only 1 worker process enabled.
- Another setting I disabled in the tab was the 'Shutdown worker processes after being idle for..'. I noticed that starting worker processes was extremely expensive in terms of CPU usage.
If your site is database driven, you may want to turn off Connection Pooling and set a low connection timeout if you're confident that your server will not have resources issues when a large number of database connections are opened. In ASP.NET, add the Connect Timeout and Pooling settings to the connection string:
Data Source=SERVERNAME;Database=DATABASENAME;Integrated Security=SSPI;Pooling=false;Connect Timeout=5;
For references, please consult the following Links
Sunday, August 09, 2009
Recently I was tasked with encrypting/decrypting some data that could be edited using ASP.NET's DetailsView control. Along with the details view control, the legacy code used a SQL Data Source to select and update everything. This means the data access layer was bypassed completely, so any business logic that dealt with encryption on this page had to use a different execution path. So, how would you encrypt and decrypt data?
The problem could be divided into two tasks:
- Create a filter to decrypt data in any bound columns.
- Edit the DetailsView event, ItemUpdating and figure out which columns needed to be updated during the event.
The first task was the easier of the two, as it just involved referencing the encryption library from the code behind page and calling the function inline in the aspx page. So, the method in the code behind page would look like something like this:
protected string Decrypt(object input)
{
return EncryptionHelper.Decrypt(input.ToString());
}
While calling the decrypt function in the DetailsView child Fields tags on the aspx page:
<asp:templatefield headertext="Decrypted Data" sortexpression="EncryptedData>
<edititemtemplate>
<asp:TextBox ID="TextBox1" runat="Server" Text='<%#Decrypt(Eval("EncryptedData"))%>' />
</edititemtemplate>
<itemtemplate>
<asp:label runat="server" text='<%# Decrypt(Eval("EncryptedData")) %>' id="Label1"></asp:label>
</itemtemplate>
</asp:templatefield>
However, decrypting the data using a templatefield tag posed another problem, extracting new values from the 'TextBox1' control. This will be solved when calling the ItemUpdating event.
After setting the DetailsView's ItemUpdating method, you will need to modify the SQL Data Source's UpdateCommand, UpdateCommandType, and UpdateParameters. The legacy code in the SQL data source used a update statement with parameters. Normally the details view control automatically detects whether to pass a null in the parameter. Instead when setting the UpdateParameters in the codebehind, the control will update the database with the parameter names entered. For example '@Parameter', will be sent in the update to the record instead of null. To solve this, you would need to automatically detect whether a value is null and then add it to the UpdateParameters list. This was done using the 'IsFieldNull' method [shown below].
Initializing the SQL data source also had another problem - because certain data needed to be re-encrypted before being sent back to the database. You will need to single out any columns that need to be encrypted. On top of that, the table being updated had over 50 columns, so typing in 50+ lines just to initialize the parameters was required, but I decided to do something else instead.
The parameters being sent to the stored procedure were identically named to the column names. Using this protocol, I decided to simply grab all the field names, which were identical to the column names and add the '@' character next during initialization. So, the method to get the column names looked like the following:
private List<string> GetColumnNameList()
{
List<string> Ret = new List<string>();
for (int i = 0; i < DetailsView.Rows.Count; i++)
{
if (!String.IsNullOrEmpty(DetailsView.Rows[i].Cells[0].Text))
{
Ret.Add(DetailsView.Rows[i].Cells[0].Text);
}
}
return Ret;
}
Initializing the update parameters looked like the following:
private void InitSqlDataSource()
{
SQLDataSource.UpdateCommand = "Update";
SQLDataSource.UpdateParameters.Clear();
List<string> paramList = GetColumnNameList();
foreach (string col in paramList)
{
if (!IsFieldNull(col))
{
if (col != "EncryptedData")
{
SQLDataSource.UpdateParameters.Add(col, "@" + col);
}
else
{
SQLDataSource.UpdateParameters.Add("EncrptedData",
EncryptionHeper.Encrypt(GetDecryptedValue()));
}
}
}
}
The method checks to make sure the data being sent isn't blank or null by executing the 'IsFieldNull' method. This method uses the 'ExtractValuesFromCell' method to grab values from the DetailsView control. This method isn't exactly well documented on MSDN, so it's not obvious at first that the passed parameter, IOrderedDictionary dictionary, is being used as a referenced parameter, not as a copy. [another weird quirk about the .NET framework] The method, GetValues, used is identical to what is covered at the article written at
David Fowler's blog.
private bool IsFieldNull(string fieldName)
{
OrderedDictionary vals = GetValues(DetailsView) as OrderedDictionary;
if (vals[fieldName] == null)
{
return true;
}
return string.IsNullOrEmpty(vals[fieldName].ToString());
}
public IDictionary GetValues(DetailsView detailsView)
{
IOrderedDictionary values = new OrderedDictionary();
foreach (DetailsViewRow row in detailsView.Rows)
{
if (row.RowType != DataControlRowType.DataRow)
{
continue;
}
DataControlFieldCell dataCell = (DataControlFieldCell)row.Cells[0];
if (dataCell.ContainingField.ShowHeader)
{
dataCell = (DataControlFieldCell)row.Cells[1];
}
dataCell.ContainingField.ExtractValuesFromCell(values, dataCell, row.RowState, true);
}
return values;
}
Thursday, August 06, 2009
It's hard to find decent BBQ, especially in larges cities. Dallas, TX has its share of franchise BBQ restaurants. If you've lived in any of the surrounding DFW areas for a few months, places like 'Sonny Bryans', 'Spring Creek BBQ', and 'Dickey's BBQ Pit' can be found easily. Each place has its own specials and caters towards a specific taste, however, I think they all fail to capture BBQ that's comparable to what you find at cook offs at rodeos or competitions in the locale. I thought this would be a problem that would never be solved, until recently.
A few weeks ago, I was invited to dine in at
Angelo's BBQ in Ft. Worth. Don't let the location or the decor of the restaurant fool you, the food in this place is extremely good. I got to try the beef brisket and the chopped beef, both were tasty and extremely tender. A quality that seems to have all but disappeared in all the franchise restaurants I've stepped into so far.
No place is perfect though. The place could use some fresh decor and maybe upgrade the serving ware from styrofoam to dishes. I also advise to get there before nightfall. The location isn't exactly in the most tourist friendly area. The restaurant location is near an industrial area, so I'm not sure how safe it is in the evenings.
Pricing isn't bad for what you get, expect to pay around $10 for a dinner platter or $5 for a sandwich. Alcohol is also served there.
All in all, if you're looking for BBQ that doesn't taste dry or like it was reheated after being stored in the freezer, I recommend trying Angelo's BBQ.
Monday, June 29, 2009
After a long hiatus, I finally got my blog back online. I've been looking into quite a few new technologies the past few months and will certainly start writing about them this week.
Thursday, March 06, 2008
I had some trouble installing proftpd today on a Linode.com VPS server. For anyone that hasn't used linode, it's a colo service that lets people lease different grades of Linux VPS servers at reasonable prices. I've been using them for 4 months with few problems.
Installing proftpd in Ubuntu is easy. All you have to do is open a Terminal window or SSH into your server and type:
sudo apt-get install proftpd
However for some reason, the Ubuntu distros mounted on Linode's servers have a quirk or two that require a few tweaks to get the server working. The first thing is whether to run the server in standalone or xinetd mode. The server won't be able access port 21 unless it's operating in standalone mode. So, set it to standalone mode.
The second quirk, which is not unique to Linode VPS servers, regards resolving the host's address. You must add the server's static address to the hosts file. For example, if your machine's name is 'johnny' and the static IP address is 79.221.23.12.
To edit your hosts file type
sudo vi /etc/hosts
Your hosts file should look like this:
127.0.0.1 localhost
79.221.23.12 johnny
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
You can determine your machine's host name by opening the file, '/etc/hostname'.
Aftewards, check your proftpd configuration. To do this, type:
sudo proftpd -td5
If you would like more information on setting up Proftpd or Ubuntu's network configuration. Please consult the following:
Sunday, February 03, 2008
For those of you enjoying your Super Bowl Sunday, I would like to extend to you a quick tip about preparing pork ribs. Baking ribs isn't that hard. Just do the following:
- Preheat the oven to 200 degrees.
- Defrost ribs. If they're still in those water / air tight plastic packages, you can defrost them quickly by submerging the plastic package in warm water.
- Season ribs and wrap the slabs in foil.
- After oven is preheated, put them on a cookie sheet and let them bake in the oven for 3 to 3.5 hours.
After baking them, just put BBQ sauce on it and sear the ribs either on a grill or hot skillet to get the BBQ sauce nice and toasted. For sauce, I usually use Original K.C. Masterpeice, and for seasoning, one of those dry rub seasonings made by Weber (yes, the same company that makes BBQ Grills).
Monday, November 26, 2007
Today I encountered my second database corruption or crash on my development machine. The problem is that my IDE drive sucks, an old IBM 75GXP. For those of you who don't remember, these drives were notorious for having all sorts of problems when they were released. Most of the problems revolved around dying after six months. Regardless, I'm still using this thing for another month.
You can tell if your database became corrupted by restarting the postmaster or typing:
sudo /etc/init.d/posrgresql-8.2 restart
The result of the command will look similar to the following:
2007-11-26 13:14:48 CST LOG: server process (PID 16312) was terminated by signal 11
2007-11-26 13:14:48 CST LOG: terminating any other active server processes
2007-11-26 13:14:48 CST LOG: all server processes terminated; reinitializing
2007-11-26 13:14:48 CST LOG: database system was interrupted at 2007-11-26 13:14:47 CST
2007-11-26 13:14:48 CST LOG: checkpoint record is at 0/965E4130
2007-11-26 13:14:48 CST LOG: redo record is at 0/965E4130; undo record is at 0/0; shutdown TRUE
2007-11-26 13:14:48 CST LOG: next transaction ID: 0/73629; next OID: 10952256
2007-11-26 13:14:48 CST LOG: next MultiXactId: 1; next MultiXactOffset: 0
2007-11-26 13:14:48 CST LOG: database system was not properly shut down; automatic recovery in progress
2007-11-26 13:14:48 CST LOG: record with zero length at 0/965E4178
Before initiating any recovery options, I recommend backing up your $PGDATA directory. The following directions will attempt to reset the transaction log, and you may loose the latest changes to your databases. The first option for recovering from a crash is to open your '/etc/postgresql/8.2/main/postgresql.conf' file and turn on the following option:
...
zero_damaged_pages = true
After saving the changes, type the following to restart postgresql:
sudo /etc/init.d/postgresql-8.2 restart
If you still see error messages, you may need to restart the entire computer in order to get postgresql to start.
After restarting postgresql, reopen the '/etc/postgresql/8.2/main/postgresql.conf' file and comment out the zero_damaged_pages option. The line should look similar to the following:
#zero_damaged_pages = true
After reading a few posts, it seems that IDE drives in general should not be used when working with databases because of corruption issues. If you are using an IDE drive, I recommend adding the following options to the '/etc/postgresql/8.2/main/postgresql.conf' file.
fsync = on # turns forced synchronization on or off
wal_sync_method = fsync
The problem with turning on fsync is that it slows down the performance of Postgresql, however, it should prevent further corruption issues with your database.
You can find more information about these configuration options in the
Postgresql 8.2 documentation.
Reference:
Thursday, November 22, 2007
The other day I had a nice chat with a colleague about the benefits and pitfalls of using an open source platform. I know this subject has been discussed deeply on various forums articles and blogs. I've personally done research for Alcatel-Lucent and the University of Texas at Dallas on the subject. For businesses, the bottom line is cost. Their are certain scenarios in a business where switching platforms, whether its open source or not, does not make sense. The scenario I discussed with my colleague revolved around the cost of switching from a Microsoft platform to a Linux platform for its custom server applications.
My colleague's business faces two main hurdles before considering a migration: the time it will take to migrate custom applications and the expenses incurred during the migration. If the current system is meeting customer needs and does not need further changes, then migrating over to an open source platform may not be the best choice, especially if the lifetime of the system is not close to expiring. Open source software is best used when continuous scaling is demanded. Because of the licensing fees incurred when scaling can become astronomical, development of the system needs to consider the benefits given to a proprietary solution over a open source solution. For example, Microsoft's Visual Studio has excellent tools for building .NET applications. If the time saved from developing on this platform is justifiable and the scaling needs do not supercede the cost saved using an open source platform, then using a Microsoft solution may be warranted.
Despite my overall excellent experience developing on the Microsoft platform, I still have a hard time recommending it as the platform of choice, mainly because of cost and time savings. These savings are not obvious to people who have not used an open platform before. One way to explain the cost and time savings incurred is the notion of barriers or hurdles towards completing a goal. When developing or deploying a system, one of the hurdles towards completion is paying for licenses, which incurs some amount of expense in time and money. If the entire application stack is free, then you eliminate that expense altogether for the rest of the system's lifetime. This allows the complete application stack to be cloned or deployed multitudes of times, whether in a test system, developer system, or production system, without incurring the expense of time and money you would encounter in a purchased product. Removing this hurdle has changed the way entire systems are deployed and general deployment of systems in both open source and commercial projects. One simple example is Debian's software distribution, 'apt'. A user can script out the default software configuration for a server with one command line. For example, installing a web server, office suite, browser, a couple of games, a couple of compilers, and a IDE with one shell command. Removing the purchasing barrier also paves the way towards completely automating the scaling of a system's infrastructure. For the small software vendor, you just can't do that on a Windows platform.
Wednesday, October 24, 2007
After a short search through the Ubuntu forums, I ran into this
post that went into details about setting up a HP printing device. After briefly reading through the instructions, I ran a utility called 'HPLIP'. 'HPLIP' is a program that will automatically download and compile all the necessary files to activate your printer. The program will ask you a few questions about your computer and request you replug-in your printer at the end of the installation. After doing this, I printed a test page and golly... it actually worked.
Thursday, October 18, 2007
The other day I successfully made a full transition from my laptop to my desktop as my primary development environment. The biggest hurdle before completing this transition was transferring and syncing documents between my two laptop and desktop. For quick file transfers, I created a network file share, or NFS, on my desktop, while mounting the drive on my laptop. For a quick overview on how to setup and mount NFS, consult this
thread on Ubuntu forums.
I also wanted to sync and compare documents from a centralized server and have the ability to compare differences between a client and a centralized 'master' copy. (think Subversion - but without all the permissions and change logging) After a quick Google search, I found a wonderful program called 'Unison'. This program will allow a user to define a master directory on a server and slave directory on a client. Master being the label for the directory where all clients compare their files and slave as clients that send new files or receive files copied from other clients to the master directory. For directions on installing unison, consult this
article on howtoforge.com.
Friday, September 07, 2007
While working on a project in ASP.NET, I needed a function that would retrieve the domain of the current url, however, I also wanted the function to also retrieve the correct ASP.NET development web server path when developing in Visual Studio. After consulting Google, I ran into
this old post on Rick Strahl's blog about the Request.Url object.
After some experimenting, I created a web page in ASP.NET 2.0 that showed what parts of the URL could be returned using different calls. Consult the following in the page load event.
protected void Page_Load(object sender, EventArgs e)
{
Response.Write("Request.Url.AbsolutePath= " + Request.Url.AbsoluteUri);
Response.Write("<br>");
Response.Write("Request.Url.AbsoluteUri= " + Request.Url.AbsoluteUri);
Response.Write("<br>");
Response.Write("Request.Url.GetLeftPart(UriPartial.Authority)= " + Request.Url.GetLeftPart(UriPartial.Authority));
Response.Write("<br>");
Response.Write("Request.Url.GetLeftPart(UriPartial.Path)= " + Request.Url.GetLeftPart(UriPartial.Path));
Response.Write("<br>");
Response.Write("Request.Url.GetLeftPart(UriPartial.Scheme)= " + Request.Url.GetLeftPart(UriPartial.Scheme));
Response.Write("<br>");
Response.Write("Request.RawUrl= " + Request.RawUrl);
Response.Write("<br>");
Response.Write("Request.Path= " + Request.Path);
Response.Write("<br>");
Response.Write("Request.ApplicationPath= " + Request.ApplicationPath);
Response.Write("<br>");
Response.Write("Request.ResolveUrl= " + ResolveUrl("~/dealer/default.aspx"));
Response.Write("<br>");
Response.Write("GetAuthorityApplicationPath= " + GetAuthorityApplicationPath());
}
private String GetAuthorityApplicationPath()
{
return String.Concat(Request.Url.GetLeftPart(UriPartial.Authority), Request.ApplicationPath);
}
The function GetAuthorityApplicationPath() is what I needed in the end to dynamically retrieve either the domain in a production environment or the development web server url while running Visual Studio (eg. 'http://localhost:1234/WebDirectory')
Sunday, September 02, 2007
Lately I've been tasked with writing an Administration interface that's very client heavy for a web application. It uses the
Extjs framework and its widgets for building the GUI,
Django +
Python for the application tier, and
PostgreSQL for the database end. We're using
Apache and
Ubuntu Server as our platform. The entire application stack is open source, so the acquisition costs for starting development is nil.
In the past few weeks, I've developed more insights into the advantages of developing on a completely open source stack. The newest pro I've discovered is the documentation and active communities in the larger and popular projects. I know a lot of MS developers moan about the lack of adequate support available on some open source projects, but it's not true of all them out there. When choosing components for your system, it's almost a given that strong community support is a requirement. Fortunately in OSS projects, the utility of a project and the general following behind it go hand in hand.
I can recommend with confidence that the community support behind our application stack (Extjs, Django, Python, PostgreSQL) for our system is strong and adequate for any web application projects that you may want to pursue
Thursday, August 09, 2007
Sometimes when working on my laptop, which has Ubuntu Desktop installed on it, I have to move it around and therefore unplug the network cable and switch to wireless or vice versa. Unplugging and replugging your laptop into a network sometimes results in the laptop's inability to renew its IP address or re-establish a connection with the internet. After a quick Google search, I found this
post about the problem.
In order to issue a command similar to 'ipconfig -renew' in Windows, open your shell and type the following:
sudo ifdown eth0
sudo ifup eth0
These two commands will renew your IP address and should fix the connection problem. However, there's a program called
'ifplugd' that monitors your network connection and automatically renews your address if this problem occurs. To install this program, open your shell and type:
sudo apt-get install ifplugd
Tuesday, August 07, 2007
Tonight I was wrapping up a deployment for a feature that was heavily dependent on javascript. After running my first batch of tests on Firefox, everything passed without a hitch. Then I ran my tests on Internet Explorer 7 and.... nothing.. literally a blank page rendered on my screen. Thus began my latest saga of fixing compatibility issues between browsers.
The first problem was that my scripts were either being downloaded and not executing, or were not downloading at all. I checked if the javascript files were being retrieved by using program called
Fiddler2. This program intercepts all HTTP requests and responses from Internet Explorer. After running the program, it confirmed that my files were being downloaded. So, this means the files were not being executed. After staring at my html code which looked like the following:
...
<body>
<script type='text/javascript' src='/file.js' />
</body>
....
I immediately remembered that Internet Explorer doesn't render javascript references unless you specify the complete tag. Why? I suspect its because I didn't specify the doctype in the html tags. So to fix that problem, I changed the script tag to look like this:
...
<body>
<script type='text/javascript' src='/file.js'></script>
</body>
However, that was not the end of my javascript problems with Internet Explorer. After running the javascript, I ran into an error regarding an undefined element in an array. My first thought was that this couldn't be. The script ran flawlessly in Firefox. After some debugging, I concluded that the Javascript engine in IE interprets text representations for arrays differently than Firefox. The evidence that lead to this conclusion came from comparing the array length returned in Firefox versus Internet Explorer. Firefox's length was 3, while Internet Explorer's was 4.
My Firefox-only-array looked something like this:
array = [ {id:1,name:'hi'}, {id:2, name:'hello'}, {id:3, name: 'greetings'}, ]
Turns out that last comma in the array throws off Internet Explorer and causes it to increment that length an extra tick. The fix would be to remove that last comma. The fixed code looks like this:
array = [ {id:1,name:'hi'}, {id:2, name:'hello'}, {id:3, name: 'greetings'} ]
Monday, August 06, 2007
After sitting down and creating a javascript client, I have to say - this really sucks. I guess moving to a heavier client tier is a natural progression of web development. Back in 1996 or so when I was still in grade school, I started messing around with HTML and basic CGI scripts for creating dynamic web pages. About 2 years later, MySQL became really popular and the first database generated web sites started to appear. The big argument back then was whether application logic should be done in the database end or in the application tier. The languages and platforms I was aware of in the open source world were Perl and PHP3 - both were equally terrible web development tools. These weren't the only open source tools out there at the time, rather quite the opposite. There was a huge plethora of different web platforms out there. Some still exist but still obscure - others just obscure. Let's just say things were weird because everyone was still trying to figure out the best way to develop web front ends. For example, it wasn't unheard of to hear SQL being used to generate HTML.
After PHP4 came out, there was an explosion in web development around the PHP, MySQL, and Apache application stack. About this time, most of the dynamic page generation got moved to the application tier. Fast forward about 6 years later to the present, we're seeing more of the page generation executed in the web browser using Javascript or some other platform built into the browser. There are numerous advantages that can be gained by moving page generation to the client: lower load on the servers, improved visual enhancements, and faster execution speed. However, developing interfaces in javascript still feels immature. To me, client side development in javascript is about equivalent to what PHP4 was back in 2001 - it does the job but has a lot of room for improvement.
This doesn't mean the javascript engines found in IE and Firefox aren't mature, its actually quite the opposite. Since Javascript has been incorporated in the browser since the 1990s, the engine is very mature and stable. It's just new requirements and forms of usage for javascript has changed since its original inception, hence why I think the current implementation seems incomplete. By incomplete, I mean issues such as browser compatibility, the occasional memory leak, and I still think development tools have room for improvement, like code completion and debugging.
The current application I'm working on uses the
ExtJs framework for generating the Admin UI and
Django as the middle tier for parsing requests between the client and the database. So far, the combination is working well.
Thursday, July 26, 2007
Today I was trying to find the location of my chat logs for gaim in Ubuntu and noticed that none of my logs were being found through the File Search program ('Places' -> 'Search for Files'). After digging through a few Google queries, I ran into a blurb about hidden folders prefixed with a period. It turns out all personal preferences user specific files for your applications are stored in directories with this notation '/home/user_name/.program_name'. So all my personal files and settings related to gaim would be stored in '/home/alex/.gaim'.
The File Search program ignores hidden files by default. In order to search hidden files, click on the 'Available Options' drop down list and select 'Show hidden and backup files' and press the 'Add' button. This should include all hidden files in your search.
To see a list of the hidden directories in your home folder, open the shell and type the following:
ls -a
By default, your command prompt should open in your home folder. So you shouldn't need to navigate to '/home/user_name/'.
Monday, July 09, 2007
The other day I was tasked with mirroring a FTP site, about 5+ gigs of files, on our local server. Mirroring directories is a fairly common task when administering servers, however, the main differences when tasked with this job are the protocols available, whether the job is bi-directional or one way, and the how fast the mirroring needs to occur.
Lucky for me, this job did not demand instantaneous sync and the job was only one way - meaning changes from a server were reflected only from one server. The biggest problem was this job was limited to using only the FTP protocol for mirroring the site. This immediately removed rsync, a popular server/client for syncing directories remotely, as an option. After a quick search through the Ubuntu forums, I stumbled upon a post that detailed several programs on mirroring an FTP site using FTP protocol only. I chose a program called 'ftpmirror'.
Ftpmirror is a program that lets a user define 'packages', which are configuration details for mirroring an FTP site, and scheduling these 'packages' to be run daily, monthly, or weekly. To install this program, I typed
sudo apt-get install ftpmirror
If you're using Ubuntu Server, the configuration files should reside in the '/etc/ftpmirror/' directory. Upon browsing through the directory, you will find a file called 'ftpmirror.cf-sample'. This file contains a few example 'packages' that can be used as a template. The user puts any active 'packages' in the 'ftpmirror.cf' file. My 'ftpmirror.cf' file looks like this:
package = alexkuo_media
ftp-server = master.alexkuo.info
ftp-user = mirror
ftp-pass = password
remote-directory = /media/pics/
local-directory = /home/deploy/media/pics
This package uses the directory '/media/pics/' as the root directory on the ftp server 'master.alexkuo.info' and uses the login/password, mirror/password, to login into the remote server. All files, directories, and subdirectories found in the '/media/pics' directory are then downloaded into the '/home/deploy/media/pics' directory on the local machine once the package is activated.
I decided to run this job once a week, so I added the package to the '/etc/ftpmirror/list.weekly' file. To do this, open the 'list.weekly' file with a text editor. Mine looks like this:
alexkuo_media
Pretty plain huh? I removed the comments that originally came with the file, so it looks pretty bare. Adding another package involves defining another package in the ftpmirror.cf file and appending the package name on a new line in one of the *.list files.
Thursday, June 28, 2007
I ran into a problem today accessing a USB Drive in Ubuntu Server. In Windows and in Ubuntu Standard, plug and play works fine. My laptop, which has Ubuntu installed, immediately recognized the USB drive once it was plugged in. I accessed its contents in the Bash shell by typing:
ls -l /media
ls -l /media/ExternalHDD
Typing the same thing in Ubuntu Server will not yield the same results. Ubuntu 7.04 server does not come with automatic mounting. After plugging in your drive, you need to run a few commands before access to the drive will be allowed. The first command is to install a program called 'pmount'. Type:
sudo apt-get install pmount
After installing the pmount, you need to figure out what the drive is called on your system. Type:
fdisk -l
You should see output that looks like this:
Disk /dev/sdf: 300.0 GB, 300069052416 bytes
255 heads, 63 sectors/track, 36481 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdf4 * 1 36481 293033601 7 HPFS/NTFS
The USB drive I am using is called '/dev/sdf4'. To mount the drive, you're need to create a directory to alias the drive and use pmount to reference the alias. To do this I type:
sudo mkdir /media/usbdrive
pmount /dev/sdf4 /media/usbdrive
Afterwards, I copied a bunch of files on the USB drive and set the ownership of the files from root to myself. I typed:
sudo cp -R /media/usbdrive/filesToCopy /home/alex/
cd /home/alex
sudo chown -R alex:alex filesToCopy
New Article
Tuesday, June 26, 2007
Tonight I deployed another installation of Ubuntu Server 7.04 Fiesty. The biggest problem I ran into was setting up the network card. Apparently, the Ubuntu setup wizard tries to contact the network in at least two points during the installation wizard. In one of the points of the installation, failure to contact a server on the internet results in the installation hanging. I got around this by running the installation while connected to my residential DHCP network and then redeploying the server to the DMZ, which requires defining a static IP.
This created another problem - How do you change the network settings from DHCP to static IP address? In order to solve this problem, you need to update your interfaces file to use an assigned static IP address and configure the server to use your network's DNS servers.
To change your network settings from DHCP to static IP, type:
sudo vi /etc/network/interfaces
The file displayed should have a line that says ' iface eth0 inet dhcp' or something similar. Change the file to look like the following:
....
....
....
auto eth0
iface eth0 inet static
address 71.164.212.40
netmask 255.255.255.0
network 71.164.212.0
broadcast 71.164.0.255
gateway 71.164.212.1
Afterwards, restart your network interface by typing:
sudo /etc/init.d/networking restart
Next, you need to configure your DNS servers. You need at least two entries. To edit your DNS entries type:
sudo vi /etc/resolv.conf
My file looked like this:
nameserver 4.2.2.1
nameserver 4.2.2.2
After saving the file, you should be able to connect to the network.
If you would like to edit your hosts file, type the following in your shell.
sudo vi /etc/hosts
You can add more references to your localhost in here. Mine looks like this:
127.0.0.1 localhost
127.0.0.1 alex-server
127.0.0.1 sandbox-server
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
If you would like to change the hostname of your server, type the following:
sudo vi /etc/hostname
After typing in your new hostname, you need to run a script to implement the change. Type:
sudo /etc/init.d/hostname.sh start
References
Sunday, June 24, 2007
The other day somebody in the neighborhood dribbled touch up paint all over my car. Judging from a bottle of paint that was left near the crime scene, I think it's lacquer-based. After pondering what to do, I consulted Google and ran into
this post. It explained that I should use something called
Goof-Off for removing the offending paint. This stuff really does work, however, it takes quite a bit of elbow grease to get the paint off.
At first I used Q-tips to remove the splotches of paint, but I found that it was easier to just use a soft paper towel. It took about 2.5 hours to remove all the dribbles and spots. The only word of caution I can offer to anyone out there when using 'Goof-off' is to take it easy on the pin stripes. This stuff is powerful enough to take the stripes off (which is bad), but not powerful enough to get through the clear coat (which is good).
After 2.5 hours of rubbing and cleaning, the paint was taken off without any problems.
Monday, June 18, 2007
I've been working on issues related to processes and PostgreSQL these past few days. Minor as they were, I think these are common issues that aren't documented very well. I recently had a problem canceling queries issued from PGAdmin. I decided to cancel the queries by looking up the process id and canceling the process thats running the query.
You can look up the process id, or the 'pid' column, by executing the following query in psql or PGAdmin:
select * from pg_stat_activity;
This query will return a list of all processes currently being run by your server. After finding the query you want to cancel, go to the BASH prompt and type:
sudo kill pid_number
Another problem that I ran into the other day was figuring out which processes a java server was running on. I did this by querying processes by name or browsing through all of the server's processes. To query processes by a name, I typed:
ps -o user,pid,ppid,command -ax | grep java
If you still can't find the process, you can browse all processes by typing:
procinfo
New Article
Tuesday, June 12, 2007
I finally deployed a Django application to a public facing Linux server. You can read all about it in a write up below. For those of you who don't know what I'm talking about, Django is another 'rapid application development' web framework that comes with all sorts of useful tools to help web application developers get their software finished faster. Some of these time saving features include an auto generated administration tool, an ORM object mapper, and a template engine.
So how does developing on Django compare with development on ASP.NET? Well, for non-MS Office related development I would say it is faster and more suitable for custom applications. Alot of this has to do with the object relational mapper, which accomplishes the same things as Microsoft's Dlinq, and the generated admin tool. These two features alone completely remove a large amount of coding that usually has to be done when creating an ASP.NET application.
I highly recommend anyone out there give Django a try. Check out
this tutorial for a quick introduction.
New Article
Sunday, June 10, 2007
In any engineering project, finding the cause of a problem can be just as hard as fixing the actual problem. Today our garage door broke, so my roommates and I tried to fix it. At first we thought the problem was caused by a broken axle on one of the track wheels. After inspecting the plate, we quickly found out that the axle was just fine.
Then we decided to inspect one of the guide wire wheels. The guide wires were definitely incorrectly configured. We found that the wire had not re-wrapped itself properly, causing the motor to push harder on the garage door when closing.
This still didn't explain why the door wouldn't close though. Stepping back to look at the garage door, we quickly realized that the two sides were not falling down at the same rate, causing one side to close faster than the other - warping one of the guide rails.
After examining the brackets that secure the track wheels to the door, we found one of the brackets had cracked. This created a harder 'push' on one side than the other when closing.
The end result is this:
At this point, we decided to call the repairman.
New Articles
Wednesday, June 06, 2007
Back in the day, I use to sling code in 2-3 hour blocks, taking nice long breaks in between sessions. This allowed my brain to think about problems deeper and come up with more elegant solutions. One of the reasons is that the house I am in turns into a noise making, hard rocking, garage band around 5 PM when all my roommates come home. Homes with children experience a similar event. In any case, I code late at night or in the mornings when no one is home. In fact, I try and code for as long as I can because I know these periods of peace and quiet don't last. I guess the 2-3 hour programming sessions will be on ice until 'quiet time' becomes the norm again.
New Article