Neil Ang (web-development)

Redirect to iPhone site with htaccess

Sat, 06 Feb 2010 01:49:00 GMT

This is my implementation of how to redirect to an iPhone optimised website using Apache rewrite rules and cookies. Although this redirect can be achieved easily through server-side scripting, websites that implement heavy page caching need to redirect at the request level.

The interaction: If the user is using an iPhone to view the site, I wanted to automatically redirect them to the iphone site, however I also wanted to allow the user to switch between the two versions of the website freely - and to remember their preference for which version of the site they wanted to view when they returned. I also didn't want to embed absolute URLs into links or append parameters to the end of urls.

So these are the rules I came up with to redirect an iphone visiting www.example.com to i.example.com. On each page of the site I include a link to switch between the two sites. Example. If the user is viewing i.example.com/about.html there would be a server relative link to "/noiphone/about.html" and on the normal site there would be a link to "/iphone/about.html" etc.

# Always redirect to the iphone site (unless there is a cookie) 
 RewriteCond %{HTTP_USER_AGENT} iPhone 
 RewriteCond %{HTTP_HOST} !^i.example.com 
 RewriteCond %{HTTP_COOKIE} !^.*no_iphone=yes.*$ [NC] 
 RewriteRule ^/(.*)$ http://i.example.com/$1 [R,L] 
 
 # User requested normal site (sets cookie) 
 RewriteCond %{REQUEST_URI} ^/noiphone/.*$ 
 RewriteRule ^/noiphone/(.*)$ http://www.example.com/$1 [R,L,CO=no_iphone:yes:.example.com:10080] 
 
 # User requested iphone site (removes cookie) 
 RewriteCond %{REQUEST_URI} ^/iphone/.*$ 
 RewriteRule ^/iphone/(.*)$ http://i.example.com/$1 [R,L,CO=no_iphone:0:.example.com:-1]

You may notice that the cookie expire time to remember their preference is set to 10080 minutes (1 week), you may want to adjust this if you don't want their preference to expire.

Please post a comment

How to create a spell checking web spider

Sat, 24 Oct 2009 10:18:00 GMT

Recently I found out my employer was paying a third-party company to regularly check the spelling on their website and send them a monthly report. So I thought I would write a simple spider that could to do the same thing and save them money.

The setup

My solution uses ruby (plus some gems) and the awesome Aspell (an open source spell checker).

First of all you will need to download and install Aspell, and an Aspell dictionary for the language you want to use. In this example I will be using the English dictionary.

To install Aspell on OS X, download and unpack the latest version, and then using terminal:

cd path/to/aspell ./configure make sudo make install

To install a dictionary, repeat the steps above with a downloaded dictionary.

You can verify that Aspell installed correctly and that the dictionaries are loaded by typing these commands in terminal:

aspell -v aspell dicts

Next you will need to install raspell, which is a gem used to interact with Aspell. You will also need another gem called spider, which will take care of the web crawling work for us. And finally, hpricot or nokogiri to handle the HTML.

sudo gem install raspell sudo gem install spider sudo gem install hpricot

The script

With that done, you can use the spider gem to crawl all the html pages on a website, then use hpricot to extract the relevant words from the page and check each one with raspell. E.g.

#!/usr/bin/env ruby require 'rubygems' require 'spider' require 'raspell' require 'hpricot' domain = 'http://www.example.com/' speller = Aspell.new('en_GB') Spider.start_at(domain) do |s| s.add_url_check do |a_url| a_url.match("^#{domain}") end s.on :success do |a_url, resp, prior_url| unless resp['content-type'].match('text/html') puts "Skipping #{a_url}" next end puts "On page #{a_url}" document = Hpricot(resp.body) document.search('head').remove document.search('script').remove document.search('link').remove document.search('meta').remove document.search('style').remove words = document.inner_text.gsub(/\s+/, ' ').strip.split(/\s/) speller.list_misspelled(words).each do |mistake| puts " * Found mistake \"#{mistake}\" perhaps you meant \"#{speller.suggest(mistake).first}\"" end end end

A few things to note about this script:

I have set the spider to use the "en_GB" dictionary (as this is what we use in Australia), but you can set it to any dictionary you installed (e.g. "en_US").
If you wanted to use nokogiri instead of hpricot, simply replace the Hpricot declaration with document = Nokogiri(resp.body) and require it at the start of the script.
Hpricot is used to strip out typically non-visible sections of the page, so that we only spellcheck the displayed words.
The spider won't search outside of the set domain.
The spider is also capable of performing a link check as it crawls.

As you can see this is only a very basic implementation, but demonstrates how easily you can create your own spelling spider!

Please post a comment

Validate an ISSN using Perl or JavaScript

Fri, 12 Jun 2009 07:57:00 GMT

Last week I wrote about validating an ISBN, so this week I thought I would post the code to validate an ISSN.

There are only a few minor differences between validating an ISSN and ISBN.

Validate ISSN in Perl

sub valid_issn { 
   my $issn = $_[0]; 
      $issn =~ s/[^\dX]//gi; 
   return if length($issn) != 8; 
   my $sum = 0; 
   my @chars = split('', $issn); 
   $chars[7] = 10 if uc($chars[7]) eq 'X'; 
   for (my $char = 0; $char < @chars; $char++) { 
     $sum += (8-$char) * $chars[$char]; 
   } 
   return (($sum % 11) == 0); 
 }

Validate ISSN in JavaScript

function isValidISSN (issn) { 
   issn = issn.replace(/[^\dX]/gi, ''); 
   if(issn.length != 8){ 
     return false; 
   } 
   var chars = issn.split(''); 
   if(chars[7].toUpperCase() == 'X'){ 
     chars[7] = 10; 
   } 
   var sum = 0; 
   for (var i = 0; i < chars.length; i++) { 
     sum += ((8-i) * parseInt(chars[i])); 
   }; 
   return ((sum % 11) == 0); 
 }

Please post a comment

How to check if an ISBN is valid in Perl or JavaScript

Thu, 04 Jun 2009 22:22:00 GMT

The cool thing about an ISBN is that its last digit is a "check digit", which validates the rest of the number. How does it work? "It is calculated on a modulus 11 with weights 10-2, using X in lieu of 10 where ten would occur as a check digit." More info.

Validate an ISBN in Perl

Here is a simple subroutine I wrote to check if a number entered is valid:

sub valid_isbn { my $isbn = shift; $isbn =~ s/[^\dX]//gi; return if length($isbn) != 10; my $sum = 0; my @chars = split('', $isbn); $chars[9] = 10 if uc($chars[9]) eq 'X'; for (my $char = 0; $char < @chars; $char++) { $sum += (10-$char) * $chars[$char]; } return (($sum % 11) == 0); }

Code would not be complete without tests!

use Test::More tests => 12; is(valid_isbn("ISBN 0843610727"), 1); is(valid_isbn("99921-58-10-7"), 1); is(valid_isbn("9971 5 0210 0"), 1); is(valid_isbn("0-8044-2957-X"), 1); is(valid_isbn("0943396042"), 1); is(valid_isbn("0-9752298-0-X"), 1); isnt(valid_isbn("ISBN 0843610723"), 1); isnt(valid_isbn("91921-58-10-7"), 1); isnt(valid_isbn("9971 5 0214 0"), 1); isnt(valid_isbn("0-8044-2957-2"), 1); isnt(valid_isbn("0943394042"), 1); isnt(valid_isbn("0-9757297-0-X"), 1);

Validate an ISBN in JavaScript

And here it is written as a JavaScript function:

<script type="text/javascript" charset="utf-8"> function isValidISBN (isbn) { isbn = isbn.replace(/[^\dX]/gi, ''); if(isbn.length != 10){ return false; } var chars = isbn.split(''); if(chars[9].toUpperCase() == 'X'){ chars[9] = 10; } var sum = 0; for (var i = 0; i < chars.length; i++) { sum += ((10-i) * parseInt(chars[i])); }; return ((sum % 11) == 0); } </script>

Please post a comment

Computer namer

Sun, 22 Feb 2009 11:30:00 GMT

Last week I was working with rails, and decided to test how much I had learnt to date. I picked a small website idea I had had aeons ago, and wanted to see how quickly and easily I could put it together in rails.

To my surprise I did better then I thought. In less then an hour I had the site, database, administration screen and mailer setup. I did a bit more work later on, and added ajax, mac style buttons and twitter.

The site - Computer namer - is a random name generator for naming your computer. It was fun to make and (hopefully) fun to use.

Please have a look at it, and let me know what you think. Also, make a suggestion for a computer name if you can think of one.

Please post a comment

Generating a calendar using XSLT

Sun, 16 Nov 2008 07:43:00 GMT

Recently I was tasked to build an interactive calendar in XSLT. I wanted the calendar to always start on a Sunday as well as have clean markup. I couldn't find any good examples on the web while I was building it, so I decided to post the base of my solution.

Insert the following variables and templates into your stylesheet:

<xsl:variable name="DisplayDate" select="date:date()"/> 
 <xsl:variable name="Year" select="date:year($DisplayDate)"/> 
 <xsl:variable name="Month" select="date:month-in-year($DisplayDate)"/> 
 <xsl:variable name="MonthName" select="date:month-name($DisplayDate)" /> 
 <xsl:variable name="NumberOfDaysInMonth"> 
   <xsl:call-template name="DaysInMonth"> 
     <xsl:with-param name="month" select="$Month" /> 
     <xsl:with-param name="year" select="$Year" /> 
   </xsl:call-template> 
 </xsl:variable> 
 <xsl:variable name="FirstDayInWeekForMonth">

  <xsl:choose>

    <xsl:when test="$Month &lt; 10">

      <xsl:value-of select="date:day-in-week(date:date(concat($Year,'-0', $Month, '-01')))" />

    </xsl:when>

    <xsl:otherwise>

      <xsl:value-of select="date:day-in-week(date:date(concat($Year,'-', $Month, '-01')))" />

    </xsl:otherwise>

  </xsl:choose>

</xsl:variable> 
 <xsl:variable name="WeeksInMonth"><xsl:value-of select="($NumberOfDaysInMonth + $FirstDayInWeekForMonth - 1) div 7" /></xsl:variable> 
 
 <xsl:template name="DaysInMonth"> 
   <xsl:param name="month"><xsl:value-of select="$Month" /></xsl:param> 
   <xsl:param name="year"><xsl:value-of select="$Year" /></xsl:param> 
   <xsl:choose> 
     <xsl:when test="$month = 1 or $month = 3 or $month = 5 or $month = 7 or $month = 8 or $month = 10 or $month = 12">31</xsl:when> 
     <xsl:when test="$month=2"> 
       <xsl:choose> 
         <xsl:when test="$year mod 4 = 0">29</xsl:when> 
         <xsl:otherwise>28</xsl:otherwise> 
       </xsl:choose> 
     </xsl:when> 
     <xsl:otherwise>30</xsl:otherwise> 
   </xsl:choose> 
 </xsl:template> 
 
 <xsl:template name="Calendar"> 
   <table summary="Monthly calendar"> 
     <caption> 
       <xsl:value-of select="$MonthName" /> 
       <xsl:text> </xsl:text> 
       <xsl:value-of select="$Year" /> 
     </caption> 
     <tr> 
       <th abbr="Sunday">Sun</th> 
       <th abbr="Monday">Mon</th> 
       <th abbr="Tuesday">Tue</th> 
       <th abbr="Wednesday">Wed</th> 
       <th abbr="Thursday">Thu</th> 
       <th abbr="Friday">Fri</th> 
       <th abbr="Saturday">Sat</th> 
     </tr> 
     <xsl:call-template name="CalendarWeek"/> 
   </table> 
 </xsl:template> 
 
 <xsl:template name="CalendarWeek"> 
   <xsl:param name="week">1</xsl:param> 
   <xsl:param name="day">1</xsl:param> 
   <tr> 
     <xsl:call-template name="CalendarDay"> 
       <xsl:with-param name="day" select="$day" /> 
     </xsl:call-template> 
   </tr> 
   <xsl:if test="$WeeksInMonth &gt; $week"> 
     <xsl:call-template name="CalendarWeek"> 
       <xsl:with-param name="week" select="$week + 1" /> 
       <xsl:with-param name="day" select="$week * 7 - ($FirstDayInWeekForMonth - 2)" /> 
     </xsl:call-template> 
   </xsl:if> 
 </xsl:template> 
 
 <xsl:template name="CalendarDay"> 
   <xsl:param name="count">1</xsl:param> 
   <xsl:param name="day" /> 
   <xsl:choose> 
     <xsl:when test="($day = 1 and $count != $FirstDayInWeekForMonth) or $day &gt; $NumberOfDaysInMonth"> 
       <td class="empty"><xsl:text disable-output-escaping="yes">&amp;nbsp;</xsl:text></td> 
       <xsl:if test="$count &lt; 7"> 
         <xsl:call-template name="CalendarDay"> 
           <xsl:with-param name="count" select="$count + 1" /> 
           <xsl:with-param name="day" select="$day" /> 
         </xsl:call-template> 
       </xsl:if> 
     </xsl:when> 
     <xsl:otherwise> 
       <td> 
         <xsl:value-of select="$day" /> 
       </td> 
       <xsl:if test="$count &lt; 7"> 
         <xsl:call-template name="CalendarDay"> 
           <xsl:with-param name="count" select="$count + 1" /> 
           <xsl:with-param name="day" select="$day + 1" /> 
         </xsl:call-template> 
       </xsl:if> 
     </xsl:otherwise> 
   </xsl:choose> 
 </xsl:template>

For this code to work, you need to be using exslt.

<xsl:stylesheet version="1.0" 
   xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
   xmlns:exslt="http://exslt.org/common" 
   xmlns:date="http://exslt.org/dates-and-times">

Once you have added the templates you can call the calendar like this:

<xsl:call-template name="Calendar" />

Finally add some CSS to the page so your calendar looks more presentable:

<style type="text/css" media="screen"> 
   table{ 
     border:1px solid #CCC; 
     border-collapse: collapse; 
   } 
   caption{ 
     font-weight:bold; 
     font-size:120%; 
     margin-bottom:8px; 
   } 
    
   td,th{ 
     text-align:center; 
     vertical-align:middle; 
     width:36px; 
     height:34px; 
     border:1px solid #CCC; 
     padding:0; 
   } 
    
   th{ 
     height:inherit; 
   } 
    
   .empty{ 
     background-color:#EEE; 
   } 
 </style>

This is only the base of the calendar I built, and you can expand this to make it browse-able. But I will leave that for another post.

Please post a comment

How to setup virtual hosts on Mac OS X 10.5 (Leopard)

Sat, 20 Sep 2008 04:29:00 GMT

Whilst developing a new project it can be convenient to setup a virtual host on your local machine. For example, if I wanted to have a local development/demonstration version of a website I'm developing (e.g. http://learnhowtojuggle.info), I could setup the following virtual host: http://learnhowtojuggle.dev.

N.B. Only follow this tutorial if you are comfortable with editing system configurations and are aware of the risks. If you have TextMate installed, you can click the file path links in these instructions to quickly open them on your local computer (More Information).

1. Modify the virtual hosts configuration

Use terminal to find and modify your virtual hosts configuration file.

mate /private/etc/apache2/extra/httpd-vhosts.conf

Append to this bottom of this file an entry for your virtual domain:

<VirtualHost *:80> ServerAdmin webmaster@learnhowtojuggle.dev DocumentRoot "/Users/neil/Sites/learnhowtojuggle" ServerName learnhowtojuggle.dev ServerAlias learnhowtojuggle.dev ErrorLog "/private/var/log/apache2/learnhowtojuggle.dev-error_log" </VirtualHost>

Make sure the DocumentRoot you enter actually exists. For convenience, I've placed mine in my Site folder. You can now save and close this file.

2. Check Apache knows about virtual hosts

Next we will need to modify the Apache configuration file to make sure it includes the virtual hosts file.

mate /private/etc/apache2/httpd.conf

Locate the following line and remove the # sign (also known as an "octothorpe" ) from in front of it.

Include /private/etc/apache2/extra/httpd-vhosts.conf

Save and close this file as well.

3. Restart Apache

For any changes to take effect on the Apache server, you will need to restart it.

sudo httpd -k restart

4. Update hosts file

Finally, the easiest way to let our browser know to use our local server when it get a request for our virtual host is to modify the hosts file.

mate /private/etc/hosts

Add the following entry:

127.0.0.1 learnhowtojuggle.dev

After you save and close this file, navigate to http://learnhowtojuggle.dev in your browser to see the newly created virtual host on your local machine.

Additional Configuration

Since I am serving my new virtual host out of my Sites folder, and I want to use my .htacess file for URL rewriting, I had to update the Apache configuration again to allow permission for this.

The configuration for my Site folder was located at /private/etc/apache2/users/neil.conf (yours will be different based on your system profile name). I then set my permissions to be pretty loose since it is only my local machine.

Example configuration:

<Directory "/Users/neil/Sites/"> Options Indexes FollowSymLinks MultiViews AllowOverride All Order allow,deny Allow from all </Directory>

After you update your directory permissions, you will need to restart apache again for these to changes to take effect.

Please post a comment

Learn How To Juggle

Sat, 15 Mar 2008 23:11:00 GMT

Last Thursday I was talking with Dean about ideas for new sites. Most of them were silly ideas and would never work, but there was one I felt I could accomplish successfully.

So over the last couple of days I've been working on a new site that teaches visitors how to juggle. I'm happy to announce it is now live. There are some pages that I'm still writing, but the basics for learning how to juggle are there.

Please check out my new site: Learn How To Juggle, and leave a comment on this post of your thoughts.

Please post a comment

A better technique to stop spam?

Fri, 01 Feb 2008 21:52:34 GMT

A couple of weeks ago I wrote an entry on how to outsmart comment spam with javascript. Although this is a good technique, it has a drawback - code maintenance. If there is a change to the mark up of the form, it is more time consuming to alter the javascript than the HTML.

To overcome this problem and still out smart spam, I've been testing a new method which uses javascript to change the form action. In it's simplest form the javascript can look like this:

<script type="text/javascript" charset="utf-8"> 
 function redirectForm(form){ 
   form.action = ''; 
   form.submit(); 
 } 
 </script> 
 
 <form action="/no-javascript" onsubmit="redirectForm(this);" method="post" accept-charset="utf-8"> 
 <p><input type="text" name="test" /></p> 
 <p><input type="submit" value="Submit" /></p> 
 </form>

N.B. The default action for Symphony forms is to submit to the same page, which is why the action is set to empty string.

If you use multiple forms that have different actions, you could expand the javascript to handle those as well:

<script type="text/javascript" charset="utf-8"> 
 function redirectForm(form, action){ 
   form.action = action; 
   form.submit(); 
  } 
 </script> 
 
 <form action="/no-javascript/" onsubmit="redirectForm(this, '');" method="post" accept-charset="utf-8"> 
 <p><input type="text" name="test" /></p> 
 <p><input type="submit" value="Submit" /></p> 
 </form>

This is just a start, there's more you can do to make this JS smarter (but I'll let you figure that out). So far this technique has worked great for me. Remember to setup a real page for the fake submission url, so that authentic users of your site without JS enabled are alerted that there submission was unsuccessful.

To get an idea on how much spam you've stopped, check your server logs to see for the number of requests to your fake submission url.

Please leave a comment with your thoughts on this technique vs the previous one, or if you know of another solution to stop spam.

Please post a comment

Outsmarting comment spam

Sun, 13 Jan 2008 08:27:00 GMT

It is often hard to find the right balance between stopping comment spam, and encouraging users to leave comments. For example, although captcha's are effective, I don't like asking my users to enter a (sometimes cryptic) code to leave a comment.

I put a lot of effort into making sure comment spam doesn't make it on my web site. However, late 2007 I started receiving comment spam from a spam-bot who wasn't selling anything, or linking back to their site, it was just nonsense that didn't get caught by my anti-spam methods.

To counter-act this annoying comment spam, I wrote a very simple piece of JavaScript that inserted the comment form dynamically. This meant that unless the spam-bot could interpret JavaScript, it wouldn't detect the embedded form on the page and attempt to spam it.

Heres what the JS looks like:

function insertCommentForm(){ 
   var parent = document.getElementById('myCommentForm'); 
 
   var form = document.createElement('form'); 
   form.setAttribute('action', ''); 
   form.setAttribute('method', 'post'); 
    
   var fieldset = document.createElement('fieldset'); 
    
   // Input: Name 
   var nameLabel = document.createElement('label'); 
   var nameText = document.createTextNode('Name '); 
   var nameInput = document.createElement('input'); 
   nameInput.setAttribute('name', 'name'); 
   nameInput.setAttribute('value', document.getElementById('myComment-name').value); 
   nameLabel.appendChild(nameText); 
   nameLabel.appendChild(nameInput); 
    
   //Input: Email 
   var emailLabel = document.createElement('label'); 
   var emailText = document.createTextNode('Email '); 
   var emailNote = document.createElement('small'); 
  emailNote.innerHTML = 'Required but never displayed'; 
   var emailInput = document.createElement('input'); 
   emailInput.setAttribute('name', 'email'); 
   emailInput.setAttribute('value', document.getElementById('myComment-email').value); 
   emailLabel.appendChild(emailText); 
   emailLabel.appendChild(emailNote); 
   emailLabel.appendChild(emailInput); 
          
          
   //Input: Website         
   var websiteLabel = document.createElement('label'); 
   var websiteText = document.createTextNode('Website '); 
   var websiteNote = document.createElement('small'); 
  websiteNote.innerHTML = 'http://'; 
   var websiteInput = document.createElement('input'); 
   websiteInput.setAttribute('name', 'website'); 
   websiteInput.setAttribute('value', document.getElementById('myComment-url').value); 
   websiteLabel.appendChild(websiteText); 
   websiteLabel.appendChild(websiteNote); 
   websiteLabel.appendChild(websiteInput); 
    
   //Input: Comment 
   var commentLabel = document.createElement('label'); 
   var commentText = document.createTextNode('Comment '); 
   var commentTextarea = document.createElement('textarea'); 
   commentTextarea.setAttribute('name', 'comment'); 
   commentTextarea.setAttribute('rows', '12'); 
   commentTextarea.setAttribute('cols', '24'); 
   commentLabel.appendChild(commentText); 
   commentLabel.appendChild(commentTextarea); 
        
   //Hidden: Section         
   var sectionHiddenInput = document.createElement('input'); 
   sectionHiddenInput.setAttribute('type', 'hidden'); 
   sectionHiddenInput.setAttribute('name', 'section'); 
   sectionHiddenInput.setAttribute('value', 'entries'); 
    
   //Hidden: Remember 
   var rememberHiddenInput = document.createElement('input'); 
   rememberHiddenInput.setAttribute('type', 'hidden'); 
   rememberHiddenInput.setAttribute('name', 'remember'); 
   rememberHiddenInput.setAttribute('value', 'on'); 
    
   //Hidden: Entry Handle 
   var rememberHiddenInput = document.createElement('input'); 
   rememberHiddenInput.setAttribute('type', 'hidden'); 
   rememberHiddenInput.setAttribute('name', 'entry-handle'); 
   rememberHiddenInput.setAttribute('value', document.getElementById('myComment-entry').value); 
    
   //Hidden: Submit 
   var submitInput = document.createElement('input'); 
   submitInput.setAttribute('type', 'submit'); 
   submitInput.setAttribute('id', 'submit'); 
   submitInput.setAttribute('value', 'Post Comment'); 
   submitInput.setAttribute('name', 'action[comment]'); 
    
   fieldset.appendChild(nameLabel); 
   fieldset.appendChild(emailLabel); 
   fieldset.appendChild(websiteLabel); 
   fieldset.appendChild(commentLabel); 
   fieldset.appendChild(sectionHiddenInput); 
   fieldset.appendChild(rememberHiddenInput); 
   fieldset.appendChild(submitInput); 
    
   form.appendChild(fieldset); 
    
   document.getElementById('myCommentWarning').style.display='none'; 
    
   parent.appendChild(form); 
 }

Since using this JS I haven't received any comment spam. And until spam-bots evolve to read JavaScript, this is a simple and effective technique to reduce time spent dealing with spam.

Also, if you use Symphony, this JS will work with your comment forms too. Contact me if you would like the updates to the XSL you will need to make this work.

Please post a comment