Our site is not a single page site. So for every page we load the same data again and again. And that’s not find. First it is stupid, second it is making the website slower and finally it increase the load on our backend for no valid reason.

localStorage for cache

To store the data we will use the HTML5 localStorage. It is available almost on every browser ( IE8 included ). It is available across tab and you can store up to 5MB ( but far less in practice ). Also the API is very simple to use, it is fast, and you get easily debug the content of the cache with the Chrome Developer tools ( under the resources tab ).

localStorage content

Some jQuery plugins have be written using localStorage to store ajax results. https://github.com/paulirish/jquery-ajax-localstorage-cache But we decided to not used them, not because we want to recreate the wheel but because the model used to invalidate the cache doesn’t work for us.

Cache invalidation

There are only two hard things in Computer Science: cache invalidation and naming things — Phil Karlton

For every cache system the most sensitive part if the algorithm which determined if the cache content is valid. With Expensify data, this task is very complex. Our data are heavily linked with each others ( receipts link to expenses on a report ). So in order to keep a simple code for our prototype, we decided that if  the user change ANYTHING, we will clear the whole cache. This is very inefficient in a “cache hit” point of view, but that way we are sure that what we are displaying is the correct version of the data.

So we know how to invalidate the cache if the user is doing an operation on the website. But what if he is using an other browser, computer or mobile device ?

For example, if you are on the receipt page with your computer and you use your Expensify mobile app to take a picture of a receipt, the web page on the browser is refreshing itself automatically and will display the receipts after few seconds. We want to keep this functionality. So we need some way to know when the cache is becoming invalid because of the action of an “external” actor.

So we store on our server the date of the latest modification on an account. If anything happened which impact the account of the user, we update this value. So before we make any request, we ask the server to return this value. If the date is more recent that the one we have stored, our cache is not valid anymore, we need to clear it.

Clear and performance

To clear our cache, we use localStorage.clear().  So we delete everything. If the user add an expense, we clear the cache for receipts, reports, etc. Everything. It is dumb and not efficient for cache hits, but we are sure that we are not going to display out of date data. Also keep in mind that this system has been designed has a proof of concept to know if this kind of strategy could help our front end performance.

With the cache, we build a statistic system which will log on the server how many request has been done, how many cache hit, number of polling, etc. And we got the good surprised to see that around 48% of the request were hitting the cache. For a stupid cache system like that it is a surprisingly good number.

In the future, we will improve these performances by creating a key-based cache expiration system. That we could delete only part of the cache instead of everything.

Ghostscript Logo

At my work I had to correct a bug related to the conversion of a PDF to a jpeg. To do that we use the convert command line tools from ImageMagick.

The server which is doing the conversion run an Ubuntu with the latest version of the package ghostscript ( 8.71).

Here is the error:

11:04 1442 ~/Desktop : convert corrup1.pdf toto.jpg
Error: /undefined in findresource
Operand stack:
–dict:8/17(L)– C2_1 1 –dict:5/5(L)– –dict:5/5(L)– Arial –dict:9/12(ro)(G)– –nostringval– CIDFontObject –dict:7/7(L)– –dict:7/7(L)– Adobe-Identity
Execution stack:
%interp_exit .runexec2 –nostringval– –nostringval– –nostringval– 2 %stopped_push –nostringval– –nostringval– –nostringval– false 1 %stopped_push 1878 1 3 %oparray_pop 1877 1 3 %oparray_pop 1861 1 3 %oparray_pop –nostringval– –nostringval– 2 1 1 –nostringval– %for_pos_int_continue –nostringval– –nostringval– –nostringval– –nostringval– %array_continue –nostringval– false 1 %stopped_push –nostringval– %loop_continue –nostringval– –nostringval– –nostringval– –nostringval– –nostringval– –nostringval– %array_continue –nostringval– –nostringval– –nostringval– –nostringval– –nostringval– %loop_continue
Dictionary stack:
–dict:1161/1684(ro)(G)– –dict:1/20(G)– –dict:75/200(L)– –dict:75/200(L)– –dict:108/127(ro)(G)– –dict:288/300(ro)(G)– –dict:22/25(L)– –dict:6/8(L)– –dict:26/40(L)–
Current allocation mode is local
Last OS error: 2
GPL Ghostscript 8.71: Unrecoverable error, exit code 1
convert: Postscript delegate failed `corrup1.pdf’: No such file or directory @ pdf.c/ReadPDFImage/634.
convert: missing an image filename `toto.jpg’ @ convert.c/ConvertImageCommand/2838.

Weird. A google of the Error: /undefined in findresource will lead you to this bug.
To sum up the installer of ghostscript is fucked up, and no new version of the package is deploy to solve it. Awesome.

ghostscript is not able to find a font ( Adobe-Identity ) and try to find a substitution font ( Arial ) but can’t find it.
So let’s correct the configuration manually.

First make sur that you have install the microsoft webfont on your system.

Then we need to edit our cidmap which is the configuration file which tell ghostscript substitution font. Mine looks like after adding the last line:
/var/lib/defoma/gs.d/dirs/fonts/cidfmap

1 /Dotum-Bold << /FileType /TrueType /Path (/usr/share/fonts/truetype/unfonts/UnDotumBold.ttf) /SubfontID 0 /CSI [(Korea1) 0] >> ;
2 /Batang-Regular << /FileType /TrueType /Path (/usr/share/fonts/truetype/unfonts/UnBatang.ttf) /SubfontID 0 /CSI [(Korea1) 0] >> ;
3 /Batang-Bold << /FileType /TrueType /Path (/usr/share/fonts/truetype/unfonts/UnBatangBold.ttf) /SubfontID 0 /CSI [(Korea1) 0] >> ;
4 /Dotum-Regular << /FileType /TrueType /Path (/usr/share/fonts/truetype/unfonts/UnDotum.ttf) /SubfontID 0 /CSI [(Korea1) 0] >> ;
5 /Arial << /FileType /TrueType /Path (/usr/share/fonts/truetype/msttcorefonts/arial.ttf) /SubfontID 0 /CSI [(Artifex) (Unicode) 0] >> ;

Make sur to use the correct path for the arial.ttf file.

Doing that produce a second error:

1:16 1442 ~/Desktop : convert corrup1.pdf toto.jpg
Can’t find CMap Identity-UTF16-H building a CIDDecoding resource.
Error: /undefinedresource in findresource
Operand stack:
–dict:8/17(L)– C2_1 1 –dict:5/5(L)– –dict:5/5(L)– Arial –dict:9/12(ro)(G)– –nostringval– CIDFontObject –dict:7/7(L)– –dict:7/7(L)– Arial CIDFont true Arial false Arial –dict:23/26(G)– Unicode.Unicode CIDDecoding true Unicode.Unicode –dict:28/48(ro)(G)– Identity-UTF16-H
Execution stack:
%interp_exit .runexec2 –nostringval– –nostringval– –nostringval– 2 %stopped_push –nostringval– –nostringval– –nostringval– false 1 %stopped_push 1878 1 3 %oparray_pop 1877 1 3 %oparray_pop 1861 1 3 %oparray_pop –nostringval– –nostringval– 2 1 1 –nostringval– %for_pos_int_continue –nostringval– –nostringval– –nostringval– –nostringval– %array_continue –nostringval– false 1 %stopped_push –nostringval– %loop_continue –nostringval– –nostringval– –nostringval– –nostringval– –nostringval– –nostringval– %array_continue –nostringval– –nostringval– –nostringval– –nostringval– –nostringval– %loop_continue –nostringval– –nostringval– 1829 13 9 %oparray_pop findresource %errorexec_pop –nostringval– –nostringval– –nostringval– –nostringval– –nostringval– –nostringval– –nostringval– –nostringval– %loop_continue –nostringval– –nostringval– –nostringval– 1829 20 10 %oparray_pop findresource %errorexec_pop –nostringval– –nostringval– –nostringval– –nostringval– –nostringval– –nostringval– %array_continue –nostringval–
Dictionary stack:
–dict:1161/1684(ro)(G)– –dict:1/20(G)– –dict:75/200(L)– –dict:75/200(L)– –dict:108/127(ro)(G)– –dict:288/300(ro)(G)– –dict:22/25(L)– –dict:6/8(L)– –dict:26/40(L)– –dict:20/26(ro)(G)– –dict:18/25(ro)(G)– –dict:16/25(ro)(G)–
Current allocation mode is global
Last OS error: 2
GPL Ghostscript 8.71: Unrecoverable error, exit code 1
convert: Postscript delegate failed `corrup1.pdf’: No such file or directory @ pdf.c/ReadPDFImage/634.
convert: missing an image filename `toto.jpg’ @ convert.c/ConvertImageCommand/2838.

The error is Can’t find CMap Identity-UTF16-H. Ghostscript can’t find this, so let’s create it !

Mine need to be created in /usr/share/ghostscript/8.71/Resource/CMap/Identity-UTF16-H

The content of this file can be find here. Or copy it from here:

% Copyright (C) 2003 Artifex Software. All rights reserved.
%
% This software is provided AS-IS with no warranty, either express or
% implied.
%
% This software is distributed under license and may not be copied,
% modified or distributed except as expressly authorized under the terms
% of the license contained in the file LICENSE in this distribution.
%
% For more information about licensing, please refer to
% http://www.ghostscript.com/licensing/. For information on
% commercial licensing, go to http://www.artifex.com/licensing/ or
% contact Artifex Software, Inc., 101 Lucas Valley Road #110,
% San Rafael, CA 94903, U.S.A., +1(415)492-9861.
%
% $Id: Identity-UTF16-H 7095 2006-10-10 10:03:37Z leonardo $
% Identity-UTF16-H CMap
% An identity mapping of UTF16 codes to CIDs.

/CIDInit /ProcSet findresource begin

12 dict begin

begincmap

/CIDSystemInfo 3 dict dup begin
/Registry (Artifex) def
/Ordering (Unicode) def
/Supplement 0 def
end def

/CMapName /Identity-UTF16-H def
/CMapVersion 1.000 def
/CMapType 1 def

/UIDOffset 0 def
% No XUID yet obtained.

/WMode 0 def

1 begincodespacerange

endcodespacerange

1 begincidrange
0
endcidrange
endcmap
CMapName currentdict /CMap defineresource pop
end
end

%%EndResource
%%EOF

And it is working !!! Woot !

Google Chrome logo

Google Chrome has the same feature since version 8 if I’m not wrong. you can see a thumbnail of your current tab and easily move to one with a smooth animation. But this feature is hidden and you need to turn a flag to enable it.

To do that, write about:flags in your address bar.

Google Chrome about flags

Then find and enable the Tab overview experiment.

Scroll down and click Relaunch Now.

That’s it ! Now use 3 fingers on your trackpad to use it !

Google Chrome Tab Thumbnail

1. download Rhino
2. download qunit-cli
3. download qunit
4. Unzip every thing
5. Go in your qunit directory, and copy the directory qunit and test
6. Paste them in qunit-cli/qunit


qunit directory structure

7. Now open your terminal / console and move into qunit-cli/rhino
8. Run the command to test that it is working

java -cp ../../rhino1_7R2/js.jar org.mozilla.javascript.tools.shell.Main -opt -1 suite.js

9. TADAAAA


qunit rhino output

10. Edit qunit-cli/rhino/suite.js to load your source files ( at the top ) and your test files ( at the bottom )

You can also change the formating of the output by editing the suite.js file and using the right functions.

Note:
You should consider the creation of an alias to run the command :

export RHINO_HOME="~/development/rhino1_7R2"
alias js="java -cp $RHINO_HOME/js.jar org.mozilla.javascript.tools.shell.Main -opt -1"
alias jsd="java -cp $RHINO_HOME/js.jar org.mozilla.javascript.tools.debugger.Main"

Links:
http://twoguysarguing.wordpress.com/2010/11/02/make-javascript-tests-part-of-your-build-qunit-rhino/
https://github.com/asynchrony/qunit
https://github.com/benjaminplee/QUnit-CLI
http://www.mozilla.org/rhino/download.html
http://docs.jquery.com/Qunit

jQuery conference San Francisco 2011 badge and Tshirt

I was there. It was ok. Some great talks and some other less good ( no offense ).

The browser action on the top right corner of the browser.

Browser Action icon

Or the page action at the right of the address bar

Page Action icon

So how can you create a toolbar ?

Well Google Chrome provide you “content scripts“. These scripts are running in the parallel of the page. So this script can manipulate the DOM of the page. So we can inject something in the page. And if this something is an iframe, we could inject a “page” without worrying about JavaScript or CSS collision. And this iframe can be host on a server or in the extension. In this case you can use the chrome.extension.getURL method to get the path of the file.

Et voilà !

jQuery UI logo

I did in the past a jQuery plugin to do that. Here is the problem I tried to solve: when you have an HTML table with thousand of lines, it is very long to display, instable, impossible to search or to sort element. Even plugins like http://tablesorter.com/docs/ can’t do th job when you have 10 000 lines. Or they use a pager which give a bad user experience.

So tried to solve this problem and I succeed. Instead of displaying every thing, I keep the data in an array in the memory. Every operation will be done on this array : sorting, filtering, search, selection, …
And I just display the number of row that the screen can display.
I check every scroll event or action on the fake scrollbar I have create to update the display.

And it works well with 90 000 lines !!! \o/

My plugin also support plugins to change the behavior or to customize the display.

But I’m sur the jQuery team is going to do something awesome ! Be sur that I will look at their progress very closely and try to be involve into this.

Terminal icon

Look at this example: I want to delete a file, but it own by an other user.

rm delete a file

I’m sur that I can delete the file, so I have to use the sudo command before the rm command.

So I have 2 solutions:

• write sudo rm mysql.sock = equal rewrite evereything
• write sudo !! = use a shortcut !!

Of course you choose the second solution: it ‘s quicker and you don’t make any mistake by retyping the command. So !! contain the last command you wrote. Let’s see in action:

!! contain the last command

And I works also if you want to add a parameter. I will use the example of the command ls which list the content of a directoy.

ls list the content of a directory

I the screenshot, I used ls then !! -a = ls -a to list all the file of the directory, Then

ls -l -a list all file of a directory with more information

!! -l = ls -a -l = ls -al

So this shortcut is very handy when you have long command to repeat.

Dashboard icon

But with some difference. In this how to, I’m going to get the icon for the Dashboard application.

First, let’s move to the directory which contain your application, which is very often the /Applications directory. Then find your application, and right click on it.

Right click on the application

Chose the option Show Package Contents. Mac OS application code and ressource are bundle inside this file. And the is the way you can access to them. A new finder window open. Move to the directory Ressources. Normally, you will find here icons files.

Package content

An icon can be an image or an icns file. If it is an icns file your are lucky, because a single file can contain several icon size for the same icon. Just double click to open it with Preview.

Content of a icns file

That way you can choose which size is the best for you need and you get an high quality image.
Just click on the image on the right and to save it just do File > Save as ….

Applications Folder Icon

Where is it store ?

First open your Finder.

Then go to : /System/Library/CoreServices and find the file CoreTypes.bundle

CoreServices Location

Right click on the file and choose Show Package Contents

Show Package Contents

A new Finder open, with “new” directories. Go to Contents/Ressources/ Et voila !

Content of the Bundle File

Here you can find all the icons that the system is using ( not icons for a specific applications). Take a look at the icon for every model of Macbook or iMac, iPhone, … they are awesome !

If you want to know more about icns files ( icon files ) or how to get the icon for a specific application, please read this article.