Archive for the ‘Web’ Category

Being a little more anonymous online

November 9, 2016

With increased State sponsored surveillance and changes in the political tenor even in formerly moderate countries more people are asking “how to I keep my communications secure?”.

Sadly for most people the complexities of being totally opaque are too onerous, add too much complexity, or are simply not worth the effort.

But there’s no need to give up totally and just roll over. There are still things you can do to keep some of your communications at least less likely to be exposed.

If you’re in a vulnerable group – which sadly in the US today potentially means people of color, (im)migrants, women and those who don’t fit into conventional gender roles – then maybe it’s time to start taking a bit more care about your digital footprints…


jsNoSpam – make it harder for bots to find your email address

March 6, 2016

If you want to put an email address on a web page, and have it human readable and easy to click on to open up in a mail client you run the risk of exposing yourself to one of the sleazier sides of the internet. Spam email. There are bots out there which relentlessly hunt down email addresses so their masters can deluge you with unsolicited commercial email (or worse, virus infections).

The best solution is to never show the email address – get your users to use a “Contact Us” form or similar so that there’s nothing for the bots to find. But sometimes you can’t do that, either because of how the pages are hosted or your client simply doesn’t want you to.

So… jsNoSpam was born. 100% javascript, so all client side and easy to insert anywhere that allows you to edit raw HTML and include javascript.

The script works by doing a number of things…

  • Requires you to encode the email addresses so they never appear in a recognizable form in the script or HTML source.
  • Supports decoding the email address back to a usable format
  • Allows you to display the de-coded address on the page, or to require a user action (mouse over, click, keyboard navigation etc) before revealing the address.

Because the email address which is inserted into the page via the script is clickable and usable like any regular mailto: link would be user inconvenience is reduced to a minimum, but the effort for a bot to scrape the address is increased and hopefully as there are enough potential variants in how the script can be applied it will keep it ahead of the game.

Here is a live demo of the code in action.

The code is hosted on GitHub, and is open source and unrestricted license (though it would be great if you find it useful if you comment here). It’s been tested in as many browsers as I can and also with assistive technologies (eg NVDA) but if you do find an issue please comment (or better yet fire off a pull request for me to incorporate your fix).

On their own, the techniques used (encoding the address, requiring user intervention etc) are not new, but hopefully combined they will produce a robust enough solution for people who need this workaround.

Keeping a-head in the clouds

November 19, 2013

One of the great things about developing on today’s cloud platforms is elastic computing. You never know what the peaks are going to look like, but you don’t want to pay for hardware you’ll only use once in a blue moon. So you opt for a dynamically adaptive scalable solution.

If you’re read any of my posts about jsErrLog (or “jsErrLog posts” if it’s still down) you’ll know that’s what I did for that service. As I’m offering it for free to anyone with a reasonable load I needed something as cost effective as possible (ie free!). When I built it I looked at Windows Azure, Amazon’s EC2, a few smaller options, Virtual Private Servers and finally settled on Google AppEngine – in common with the others it offered a number of options for programming languages and data storage but the big bonus was a no-nonsense the free tier.

Sometimes however things don’t go quite as planned…


Lazy developers make for bad user experiences

March 18, 2013

As a developer I can appreciate that dealing with user input is a pain. Dealing with anything messy humans do is always more annoying than handling nice clean inputs from an API. Developers and designers are human too, and they should think about the experiences they are creating, and how a little bit of consideration for the user can turn a frustrating process into a moment of delight.

  • Required fields: Indicate visually when a field is required, and ask yourself if the field is actually required for what the user is trying to do (delight them and they’ll come back and share more information incrementally). Especially in a world of security leaks I like to minimize what I share and you should help with that.
  • Formatting (phone and credit card numbers) is irrelevant: Should I enter my cell as (425)-555-5555, 4255555555, 425 555 5555 or something else? Actually all of those should be valid as it doesn’t take much effort to strip out spaces, dashes and brackets when you’re validating a credit card or phone number. If you need a particular format for your database or display then re-format it… but don’t force the user to comply with a rigid structure to make your life easier.
  • Don’t be redundant: Don’t make me tell you what type of credit card I’ve entered the number for. Using a simple issuer lookup you can tell me if I just entered an Amex or a Mastercard. If you need me to write an look-up API for you I will, just leave a note in the comments.
  • Passwords are a pain to remember: Just because you think the password rules on your site are obvious (at least one capital, one digit, only special character is an underscore and it must start with a different letter than your username) users have lots of passwords. Give them a reminder next to a where they have to enter it what those arbitrary rules are, ideally on initial entry and as an absolute must if validation fails.
  • Don’t ask me the same thing twice: In the US a ZIP code can tell me the City and State. Same in Australia or New Zealand or the UK and pretty much anywhere else. Can anyone explain to me why I have to enter both 90210 and Beverly Hills, California on a million forms? By all means display the City/State for me to confirm but don’t waste my time asking me to do a computers job. That thing I said about look-up APIs earlier, still true
  • Don’t be forgetful: Computers are good at remembering stuff, if developers are not being lazy. If I fill in a field or check a box on a form and something goes wrong with validation the only field I should reasonably be expected to re-enter is the password (and if you validate that and it passes assume I know my password and don’t make me rekey those asterisks again). If I checked “accept Ts&Cs” or “Don’t email me crap” the first time… I probably meant it so don’t forget it because I didn’t get my phone number in exactly the format you like.
  • On-the-fly, context sensitive validation is awesome: Make use of onchange and onblur events and Ajax to check each field as I go to save the user scrolling up and down a page to find what failed. Basic validation, like credit card checksums, for fields that are easy to miskey should not require a full form submission
  • When things go wrong, show me: When you finally get to a full round trip validation and have to show the user some errors you need correcting don’t just bundle some obscure messages at the top of the page – use visual cues and clear explanations to guide them to get it right

By making the process simple and eliminating points where the user can stumble your helping ensure that your form is not a roadblock where the user might get frustrated and abandon the process. When you go to the supermarket you look for the shortest line, or the easiest way to checkout, and you get frustrated if the process isn’t smooth. It’s just the same on the Web.

Even if you think you’ve gone beyond the things I mention above have you gone far enough? Are you watching your logs and other telemetry to see what fields users are stumbling on? Could you streamline the process further?

Your challenge: As designers and developers you should embrace the opportunity to streamline your users experience, and use every tool at your disposal to deliver a great user experience.

Spam and security

March 5, 2013

With a series of high profile security breaches in recent weeks (Twitter, Evernote, LinkedIn and others) the obvious concern is that the attacker has access to your account. In some cases it’s more than that. (more…)

Mobile Browsers getting better

December 14, 2011

After struggling for years with sub-par browsers on small screen devices Apple did a good job raising the bar with Safari on the iPhone and now iPad. Google for some reason stumbled a little out of the gate with the browser on Android – rather than take their existing and proven Chrome they delivered an older and less capable core and it looks like only now with the 4.x generation devices that they’re finally starting to improve things (though that doesn’t help the 80-90% of their existing user base who will never get an upgrade to the latest goodness)

With the initial release of Windows Phone 7 it seemed like Microsoft had fallen into the trap of not treating mobile browsing as a first class experience and they shipped an IE7-like browser. That changed however with the release of v7.5 “Mango” which brought the full capabilities of IE9 (arguably the most standard adherent HTML5 browser) to the platform.

With people paying more attention to the mobile browser evolution and digging into capabilities (such as this review from Sencha) and arguing that the mobile web is not going away any time soon it would be great to see how the "Mango" with IE9 stacks up in these tests against the other two… Would be great to see Sencha or similar run their tests against all three? So far playing around with IE9 in Mango, it seems to do a really good job in most situations but I’ve not drilled down to quite the level of detail that they have yet.

playing a .pls source in the video tag

October 6, 2011

One thing that had been bugging me with the <audio> tag in HTML5 was that it while it was great at handling single files it doesn’t cope very well with being pointed at a shoutcast/icecast .pls file.


A bit of digging and I found that if you broke open the .pls file and used the actual stream source as the <audio src=”?”> parameter then it was quite happy to play the stream.


As a .pls file can contain a number of source files (subsequent items in a playlist, failovers in case one server isn’t running etc) the code also needed to cope with either an error or the selected stream coming to an end – luckily that isn’t too difficult with the event model supported by the audio tag.


I’ve shared a first cut of the code here on GitHub so feel free to edit/improve and share – for instance my code assumes that the NumberOfEntries parameter in the .pls file is the second line (in some instances it’s the last but one line) so there’s room to make it more robust.


Obviously it can’t perform magic … unless the stream is using a codec that your browser supports it will simply error out, but it does provide a fairly simple way to extend the capabilities of the <audio> tag

a friendly bear to help with VPN tunnels

August 24, 2011

If you want to keep your internet traffic secure when using public WiFi or have a desperate need to pretend you’re in a different country to access an online service you’ve probably tried a Virtual Private Network (VPN) service like StrongVPN. As powerful as most of these services are they’re not exactly user friendly and for a casual user they can work out quite expensive.

TunnelBear hopes to change that with an easy to install, easy to configure and, above all, easy to use app. It also starts at a pretty great price – Free!

Currently available for Windows and OSX (hopefully Linux and iOS to follow) it’s a simple install that delivers both the simple dashboard app and the network drivers needed for VPN support and then it’s a case of fire up the dashboard, decide if you want to appear as a UK or US user and hit the “on” button to switch your network connection over to using the VPN. You can change locations or de-activate the VPN just by tapping a button.

For free users you get a monthly allowance of 500MB which should be enough for simple casual needs (and they run promotions where you can bump that allowance up). If you need a bit more – in fact, unlimited bandwidth and double the level of encryption on your connection – then they have a “Giant” plan for US$4.99/mo – less than the price of a coffee at the Starbucks where you’d want to be running this.

Looking forward to seeing this for Linux so I can add it to my bootable USB Key solution.

Let Frebber make your FREB files easier to handle

June 16, 2011

If you have used IIS for any length of time you have probably come across the term FREB. If you don’t know what it is then you should read this great introduction to Failed Request Tracing in IIS. It’s applicable to IIS7 and above and is a great tool.

At a high level FREB produces an XML file containing details of errors you are interested in – you specify the error code you want to trap, the execution time threshold or a number of other filters – and provides a wealth of information about what was happening under the covers in IIS.

The problem with FREB Tracing though is that it’s very easy to end up with a folder containing hundreds or even thousands of error reports – all named a variant on fr000123.xml – and you have no way to quickly tell which where the ones with details of 401.3 errors, or which ones failbed because they took more than 5 seconds to execute.

Well, thanks to the wonders of powershell there’s now a simple solution.

Frebber scans the output directory where your FREB logs are stored and copies the files into a new subdirectory (called .Frebber of course) while at the same time renaming the files based on the nature of the error report they contain.

For instance fr000012.xml may contain details of an HTTP 415 error and took 2571ms to execute, so the file would be renamed 415_STATUS_CODE_2571_fr000012.xml

It’s a fairly simple script and if you have a look at the XML format inside a FREB report you’ll be able to see how to adapt it quickly to your particular needed. Meanwhile feel free to use the example below, and I’d love to hear any comments or suggestions in the comments.

Oh, it does make one pretty big assumption… that your FREB files are going to the default directory. If that’s not that case then you will need to modify that line (I might get around to making the script more complete and add parameter for source and destination directories and some renaming selection criteria but right now this works pretty well for me

$frebDir = "c:inetpublogsFailedReqLogFilesW3SVC1"
echo "Frebbering...."
$fileEntries = Get-ChildItem $frebdir*.* -include *.xml;
$outDir = $frebDir + ".Frebber"
# Create the directory for the Frebberized files
$temp = New-Item $outDir -type directory -force
# copy in the freb.xsl so you can still view them
Copy-Item ($frebDir+"freb.xsl") $outDir
$numFrebbered = 0
foreach($fileName in $fileEntries) 
    [System.Xml.XmlDocument] $xd = new-object System.Xml.XmlDocument
    $frebFile = $frebDir + $;
    $nodelist = $xd.selectnodes("/failedRequest")
    foreach ($testCaseNode in $nodelist) 
        $url = $testCaseNode.getAttribute("url")
        $statusCode = $testCaseNode.getAttribute("statusCode")
        $failureReason = $testCaseNode.getAttribute("failureReason")
        $timeTaken =  $testCaseNode.getAttribute("timeTaken")
        $outFile = $frebDir + ".Frebber" + $statusCode + "_" + $failureReason + "_" + $timeTaken + "_" + $;
        Copy-Item $frebFile $outFile
        $numFrebbered +=1
echo "Frebbered $numFrebbered files to $outdir."

jsErrLog: now alerts via XMPP

June 13, 2011

Although it’s nice to know that the jsErrLog service is sitting there recording errors that your users are seeing it does put the onus on developers to remember to check the report page for their URL to see if there have been any issues.

To make things a little more pro-active registered users can now connect to an XMPP (Google Chat) client (eg Digsby) and every time there’s a new error reported the bot will send you an alert.

Because you might get a flurry of messages if you deploy a version and there’s an error, or a 3rd party component has a problem the bot also listens to a set of messages so it’s easy to suspect the alerting (or turn it back on when the problem has been fixed).

At the moment there a few restrictions:

·         alerts have to match a specific URL

·         for a given user all alerts are turned off/on (no per URL granularity)

·         alerting is only available to users who’ve made a donation or promoted jsErrLog

The reason for the first one is a limitation with the way AppEngine lets me query data (unlike SQL the GQL query language does not support the CONTAINS or LIKE verbs)… I’m looking for a solution to that

The second is a feature that I plan to add soon depending on demand.

The third… at the moment it takes a little bit of setup to add new users so I’m adding it as the first freemium feature though this may change. If you want that enabled please let me know the URL you are monitoring and your Google Chat ID and I’ll let you know what else you need to do to enable it…