Injecting Current Git Commit ID Into Java
28 Dec 2009
Following is a quick hack to inject the last commits id into the application, this ant task will create a file called "commit-id" in your build directory which will contain the last commits id,
<target name="commit-id" depends="">
<exec executable = "git" output="${build.dir}/commit-id">
<arg value = "rev-parse" />
<arg value = "HEAD" />
</exec>
</target>
Now from any where in your application we can read the commit id and log it or display it,
try{
InputStream s = some.class.getResourceAsStream("/commit-id");
BufferedReader in = new BufferedReader(new InputStreamReader(s));
logger.info("Build from commit " + in.readLine());
}catch( Exception e ) {
logger.warning(e.toString());
}
Another alternative is to have a properties file that your application reads and use the "replace" task to replace a variable with the commits id.
GPS Traces Using World Wind
25 Dec 2009
I have a lot of GPX files lying around from various biking trips. I don't have much use for them other than loading them in to Google Earth and brag about being there and done that, so I thought I would parse the file myself and paint the route on the World Wind.
GPX files are XML files, containing the coordinates you have been to, if you draw a line through these points you can rebuild the route you have taken.
(defn points [f]
(let [data (zip/xml-zip (xml/parse f))
trkpt (xml-> data :trk :trkseg :trkpt)]
(map #(vector (attr % :lat)
(attr % :lon)
(first (xml-> % :ele text))) trkpt)))
gpx=> (take 2 (points "data.gpx"))
(["41.011934" "29.196977" "154.909546"]
["41.010155" "29.195925" "150.583618"])
This will return a sequence of vectors, each containing three items, latitude, longitude and elevation of the point. To draw a path on to the map World Wind provides two classes SurfacePolyline and Polyline. SurfacePolyline will draw a flat line on to the surface between coordinates where as Polyline draws a GL line between positions, which includes elevation data.
(defn surface-polyline [points]
(let [list (ArrayList.)]
(doseq [p points]
(.add list (LatLon/fromDegrees (Double. (p 0)) (Double. (p 1)))))
(doto (RenderableLayer.)
(.addRenderable (SurfacePolyline. list)))))
To create a SurfacePolyline, we need to create a list of LatLon objects and pass that to the SurfacePolyline constructor at this point you can also set a color for the line then we create a RenderableLayer and add our SurfacePolyline to it, this final layer is what will be added to the World Wind canvas.
(defn polyline [points]
(let [list (ArrayList.)]
(doseq [p points]
(.add list (Position.
(LatLon/fromDegrees (Double. (p 0)) (Double. (p 1)))
(Double. (p 2)))))
(doto (RenderableLayer.)
(.addRenderable (Polyline. list)))))
Process of creating a Polyline is the same, the only difference is instead of building a list of LatLon objects, we build a list of Position objects which includes the elevation data, again we get a layer that is ready to be added to the world. You can grab the code here.
Hello World Wind
23 Dec 2009
World Wind is a virtual globe, developed by NASA, it gives us the ability to add Google Earth like functionality to our applications.
In my previous post Mashups Using Clojure, I produced a HTML file and used Google Maps to visualize the data, with world wind we can integrate the map directly into the application.
I am going to be using the same parsing routines that I used for the Mashups Using Clojure post so I am going to skip explaining those and jump directly to World Wind stuff.
First you need to get the latest SDK, unzip it and copy the jar and library files in to your Java extensions folder, if you put them somewhere else on your classpath don't forget to set your library path to point to the native libraries.
(defn world []
(Configuration/setValue AVKey/INITIAL_LATITUDE 39.3113)
(Configuration/setValue AVKey/INITIAL_LONGITUDE 32.8038)
(Configuration/setValue AVKey/INITIAL_ALTITUDE 1000000)
(doto (WorldWindowGLCanvas.)
(.setModel (WorldWind/createConfigurationComponent
AVKey/MODEL_CLASS_NAME))))
The component that holds the world is called WorldWindowGLCanvas, you don't need to set any initial values but you need to set the model.
(defn goto-pos [world lat long elev]
(let [position (Position. (LatLon/fromDegrees lat long) (* elev 10000))
view (cast BasicOrbitView (.getView world))]
(.goTo view position (* elev 10000))))
If later on you want to change the orientation of the map, fly to a different location, you can call goTo method of the view but beware, if you try to use it before the map is shown in a frame it won't work, configure initial values instead.
(defn icon [quake]
(doto (UserFacingIcon.
"icon.png" (Position.
(LatLon/fromDegrees (Double. (:latitude quake))
(Double. (:longitude quake))) 0.0))
(.setToolTipText (apply str (interleave quake (repeat " "))))))
In order to mark positions on the map, world wind provides UserFacingIcon class, it takes an image for the icon and the coordinates to place the icon.
(defn icon-layer [icons]
(let [layer (IconLayer.)]
(doseq [icon icons]
(.addIcon layer icon)) layer))
After creating the icons you need to put them on a IconLayer,
(defn icon-layer [icons]
(let [layer (IconLayer.)]
(doseq [icon icons]
(.addIcon layer icon)) layer))
By default icons don't work like markers in Google Earth, you can't click on them, to capture events that are happening on the map, we need to install a SelectListener and check if the event occurred on a UserFacingIcon, if so we toggle it's tooltip.
(defn select-listener []
(proxy [gov.nasa.worldwind.event.SelectListener] []
(selected
[e]
(let [object (.getTopObject e)]
(if (= (.getEventAction e) SelectEvent/LEFT_CLICK)
(if (instance? UserFacingIcon object)
(.setShowToolTip object (not (.isShowToolTip object)))))))))
Thats all it takes to create our mashup, we need to create the world, create a icon for each earthquake, place them on a layer and add it to the world,
(defn frame []
(let [world (world)
layers (.getLayers (.getModel world))
earth-quakes (map #(icon %) (eartquakes))]
(.add layers (icon-layer earth-quakes))
(.addSelectListener world (select-listener))
(doto (javax.swing.JFrame.)
(.add world)
(.setSize (Dimension. 400 400))
(.setAlwaysOnTop true)
(.setVisible true))))
Full code listing can be found here.
ABRKMZZW8ASP
Cloning Pong Part 2
20 Dec 2009
This write up is part two of cloning pong, we will modify the pong game we created in part one, to be able to control it with knobs, like the original using Arduino.
To get Java communicate via Serial Port there are two options, JavaComm API and rxtx API. Arduino uses rxtx so I went with that. Navigate to you Arduino installation folder, for Mac OS X navigate into the Arduino.app,
open /Applications/Arduino.app/Contents/Resources/Java/
From the folder copy,
- RXTXcomm.jar
- librxtxSerial.jnilib
to somewhere on your classpath. Do not compile rxtx yourself, when I compiled it manually and put it on my classpath, Arduino IDE could not connect to the board, use the one supplied with your IDE, Mac OS X users make sure you are using 32 bit Java 1.5.0 otherwise you will get class not found exceptions.
Open Arduino IDE and make a note of your serial port name, on my machine its,
(def arduino-port "/dev/tty.usbserial-A6008nhh")
In order to open the serial port for I/O we need to get a port identifier from the API,
(defn port-identifier []
(let [ports (CommPortIdentifier/getPortIdentifiers)]
(loop [port (.nextElement ports)
name (.getName port)]
(if (= name arduino-port)
port (recur (.nextElement ports) (.getName port))))))
Using the identifier we can open the port for I/O, make sure baud rates match between Clojure and Arduino,
(defn open-port []
(doto (.open (port-identifier) "pong" 10000)
(.setSerialPortParams
9600 SerialPort/DATABITS_8 SerialPort/STOPBITS_1 SerialPort/PARITY_NONE)
))
Now everything is set to read from the port,
(defn poll-port [p]
(with-open [in (BufferedReader. (InputStreamReader. (.getInputStream p)))]
(.readLine in)))
Arduino will map knob reading between 0 and board size and will send coordinate pairs every 50 milliseconds, when poll-port is called, it will return,
"100:200"
meaning player one is at y 100 and player two is at y 200. We modify move-player function to check serial for input, parse the coordinates returned and set the player positions accordingly,
(defn move-player [coords player]
(try
(let [p1-y (BigInteger. (first (re-split #":" coords)))
p1-x (:x (:1 @player))
p1-src (:src (:1 @player))
p2-y (BigInteger. (second (re-split #":" coords)))
p2-x (:x (:2 @player))
p2-src (:src (:2 @player))]
(dosync (alter player merge
{:1 {:x p1-x :y p1-y :src p1-src }
:2 {:x p2-x :y p2-y :src p2-src}})) )
(catch Exception e)))
Because we don't listen for key inputs anymore, we need to modify actionPerformed call and move players before every repaint,
(actionPerformed
[e]
(move-player (poll-port serial) player)
(move-ball ball)
(wall-collision ball player)
(player-collision ball player)
(.repaint this))
Thats all the modification needed in our pong game. Code for the Arduino is even simpler,
const int boardheight = 400;
const int p1Potpin = 0;
const int p2Potpin = 1;
void setup(){
Serial.begin(9600);
}
void loop(){
int p1 = analogRead(p1Potpin);
int p2 = analogRead(p2Potpin);
Serial.print(map(p1,0,1024,0,400));
Serial.print(":");
Serial.println(map(p2,0,1024,0,400));
delay(50);
}
We just read read the potentiometers every 50 milliseconds, map the reading between 0 and board height and write it to serial. Hardware setup looks like the following, fritzing project is also available check below for a list of files.
Files
Cloning Pong Part 1
19 Dec 2009
This will be a two part write up, part one will cover building a simple pong game in Clojure, part two will cover, interfacing it with Arduino, so that we can control the game using knobs like the original.
If all you want it to play some Pong, source code for the game can be found here. I wanted to keep the code simple so it has some flaws such as the ball only goes in 45 degree angles and you can only hit the ball with the front of the pad. If you want a perfect clone, flaws can be fixed at the expense of making the code a little bit more complex.
Two references are used in the game, one for the ball and one for the players,
{:x 400, :y 200, :size 8, :dir {:x -1, :y 1}}
Reference for the ball contains it's x and y coordinates on the game board and the direction it is travelling.
{:1 {:x 10 :y 200 :src 0}
:2 {:x 400 :y 300 :src 0}}
Reference for the players contains positions of the pads and keeps track of the scores. Thats all the mutable state the game has.
Lets go over the important bits of the code,
(defn move-player [p dir player]
(let [x (if (= p :1) (:x (:1 @player)) (:x (:2 @player)))
y (if (= p :1) (:y (:1 @player)) (:y (:2 @player)))
delta (* 3 (if (= dir :up) (:up step) (:down step)))]
(dosync (alter player merge
{p {:x x :y (- y delta) :src (:src (p @player))}}))))
(move-player :1 :up
(ref {:1 {:x 10 :y 200 :src 0} :2 {:x 400 :y 300 :src 0}}))
pong=> {:1 {:x 10, :y 185, :src 0}, :2 {:x 400, :y 300, :src 0}}
When move-player is called, depending on the player and direction value passed, it will calculate the new x and y coordinates and update the player reference.
(defn place-ball []
(let [x (if (= (rand-int 2) 0) -1 1)
y (if (= (rand-int 2) 0) -1 1)]
{:x (int (/ (:w board-size) 2))
:y (int (/ (:h board-size) 2)) :size 8 :dir {:x x :y y}}))
Every time a player scores, position of the ball is reset, it will be placed in the middle of the board with a random direction.
(defn move-ball [ball]
(let [x (cond (= 1 (:x (:dir @ball))) (+ (:x @ball) (:right step))
(= -1 (:x (:dir @ball))) (+ (:x @ball) (:left step))
:else (:x @ball))
y (cond (= 1 (:y (:dir @ball))) (+ (:y @ball) (:up step))
(= -1 (:y (:dir @ball))) (+ (:y @ball) (:down step))
:else (:y @ball))]
(dosync (alter ball merge {:x x :y y}))))
Before every repaint of the game board, we move the ball in the direction it is going and update its reference.
(defn wall-collision [ball player]
(cond
(<= (:x @ball) 0) (do (inc-score :1 player)
(dosync (ref-set ball (place-ball))))
(>= (:x @ball) (:w board-size)) (do (inc-score :2 player)
(dosync (ref-set ball (place-ball))))
:else
(let [dir-x (:x (:dir @ball))
dir-y (cond (<= (:y @ball) 0) 1
(>= (:y @ball) (:h board-size)) -1
:else (:y (:dir @ball)))]
(dosync (alter ball merge {:dir {:x dir-x :y dir-y}})))))
Here we check for collisions, if the ball hits the right wall, we increment player one's score and reset the ball, if the ball hits the left wall we increment player two's score and reset the ball, if the ball hits the upper or the lower wall, we change it's y direction which will cause it to bounce from the wall.
(defn player-collision [ball player]
;;p1 collision
(if (and (= (:x @ball) (+ (:x (:1 @player)) (:w pad-size)))
(>= (:y @ball) (:y (:1 @player)))
(<= (:y @ball) (+ (:y (:1 @player)) (:h pad-size))))
(do (dosync (alter ball merge {:dir {:x 1 :y (:y (:dir @ball))}}))
(beep)))
;;p2 collision
(if (and (= (:x @ball) (:x (:2 @player)))
(>= (:y @ball) (:y (:2 @player)))
(<= (:y @ball) (+ (:y (:2 @player)) (:h pad-size))))
(do (dosync (alter ball merge {:dir {:x -1 :y (:y (:dir @ball))}}))
(beep))))
Besides walls, ball collides with the player pads, when the ball collides, with a player we keep it's y direction but change it's x direction to send it in the opposite direction.
(defn board [player ball]
(proxy [JPanel ActionListener KeyListener] []
(paintComponent
[g]
(proxy-super setOpaque false)
(proxy-super paintComponent g)
(let [quarter (int (/ (:w board-size) 4))]
(doto g
(.setColor Color/black)
(.fillRect 0 0 (:w board-size) (:h board-size))
;;ball
(.setColor Color/white)
(.fillOval (:x @ball) (:y @ball) (:size @ball) (:size @ball))
;;pads
(.fillRect (:x (:1 @player)) (:y (:1 @player))
(:w pad-size) (:h pad-size))
(.fillRect (:x (:2 @player)) (:y (:2 @player))
(:w pad-size) (:h pad-size))
;;scores
(.setFont (Font. "arial" Font/PLAIN 40))
(.drawString (str (:src (:1 @player))) quarter 50)
(.drawString (str (:src (:2 @player))) (* quarter 3) 50)
;;dashed line
(.setStroke (BasicStroke. 3 BasicStroke/CAP_BUTT
BasicStroke/JOIN_BEVEL 0
(float-array [12 12]) 0))
(.drawLine (int (/ (:w board-size) 2)) 0
(int (/ (:w board-size) 2)) (:h board-size)))))
(actionPerformed
[e]
(move-ball ball)
(wall-collision ball player)
(player-collision ball player)
(.repaint this))))
paintComponent just draws the board nothing too fancy, actionPerformed however contains the game logic, every 50 millisecond we move the ball, check for a wall collision, check for a player collision then repaint the game board. All that's needed now, is to put the game board in a frame and start it's timer.
(defn pong []
(let [ball (ref (place-ball))
mid-y (int (/ (:h board-size) 2))
player (ref {:1 {:x 10 :y mid-y :src 0}
:2 {:x (- (:w board-size) 20) :y mid-y :src 0}})
brd (board player ball)
timer (Timer. 50 brd)]
(doto (JFrame.)
(.add brd)
(.setTitle "Pong!")
;;(.setDefaultCloseOperation JFrame/EXIT_ON_CLOSE)
(.setLocationRelativeTo nil)
(.setAlwaysOnTop true)
(.setResizable false)
(.setSize (java.awt.Dimension. (:w board-size)
(+ 22 (:h board-size))))
(.addKeyListener (key-listener brd player))
(.setVisible true))
(.start timer)))
Mashups Using Clojure
17 Dec 2009
Turkey sits on top of the North Anatolian Fault, so earthquakes are something that happens a lot here. This mashup was one of the first problems I tackled in Clojure, I was going to throw it away but figured it may be use to someone else or to me later on.
Kandilli observatory and earthquake research institute releases latitude, longitude and magnitude information for the the last 200 earthquakes, so I grabed the list of earthquakes from their site, and pin them on Google Maps, that way I got a nice visual representation.
The data we are interested in looks like the following,,
Date Time Latit(N) Long(E) Depth(km) MD ML MS Region
---------- -------- -------- ------- ---------- ------------ -----------
2009.12.17 04:56:13 37.4865 35.6947 4.0 3.4 -.- -.- KOZAN
So we begin with a structure to hold the earthquakes,
(ns eartquake
(:use clojure.contrib.str-utils)
(:use clojure.contrib.duck-streams)
(:use clojure.contrib.prxml)
(:import (java.net URL)
(java.io BufferedReader InputStreamReader)))
(defstruct earth-quake
:date :time :latitude :longitude :depth :md :ml :ms :location)
Then we need to grab the page containing the data we are interested in,
(defn fetch-url[address]
(let [url (URL. address)]
(with-open [stream (.openStream url)]
(let [buf (BufferedReader.
(InputStreamReader. stream "windows-1254" ))]
(apply str (interleave (line-seq buf) (repeat \newline )))))))
We take each line and split it into nine parts not ten because last column is variable length, and apply the struct on it, that way we get a vector of earthquake structures.
(defn parse [data]
(map
#(apply struct earth-quake (re-split #"\s+" % 9))
(re-split #"\n+" data)))
The data portion of the page we are interested in begins just before the first earthquake with the dashes up to the closing pre tag, a simple regex will match it, and pass it to parse,
(defn eartquakes []
(let [page (fetch-url "http://www.koeri.boun.edu.tr/scripts/lst9.asp")
data (re-find #"(?s)------------ -----------\n(.*?)</pre>" page)]
(parse (data 1))))
We have all the data we need to build the mashup, this will be kinda reverse but I am going with the order functions are defined so when appended one after another it will be working example, markers function will build a vector of createMarker calls for each earthquake, createMarkers function is defined in Javascript which will just put a pin to a given coordinate with the string representation of the structure as the description,
(defn markers [earth-quakes]
(map
#(str "createMarker(" (:latitude %)","(:longitude %)
",'" (apply str (interleave % (repeat "<br>"))) "');")
earth-quakes))
This will result in 200 createMarker calls, which will be placed into the resulting web page. Following template is bare minimum that is required to show a google map with the addition of the createMarkers function,
(defn template[earth-quakes]
[:html
[:head
[:meta {:http-equiv "Content-Type"
:content "text/html; charset=utf-8"}]
[:title "Earthquake Mashup"]
[:script
{:src (str
"http://maps.google.com/maps?file=api&v=2&sensor"
"=false&key=abcd") :type "text/javascript"}]
[:script {:type "text/javascript"}
[:raw!
(str"
function initialize() {
if (GBrowserIsCompatible()) {
var map = new GMap2(document.getElementById(\"map_canvas\"));
map.setMapType(G_SATELLITE_MAP);
map.setCenter(new GLatLng(39.3113, 32.8038), 7);
map.setUIToDefault();
function createMarker(lat,long,desc) {
var point = new GLatLng(lat,long);
var marker = new GMarker(point);
GEvent.addListener(marker, \"click\", function() {
marker.openInfoWindowHtml(desc);
});
map.addOverlay(marker);
}")
(apply str (markers earth-quakes))
(str "}
}")]]]
[:body {:onload "initialize()" :onunload "GUnload()"}
[:div {:id "map_canvas" :style "width: 100%; height: 100%"}]]])
This will switch the map to satellite view, center it roughly on the center of Turkey and place the markers on the map, all that is needed now is to spit out a html file containing all the data,
(defn mash-up []
(spit "mash.html" (with-out-str (prxml (template (eartquakes))))))
Of course it is not practical to click on a HTML file every time we want to view the data so we need a function to automatically show the spited file, this will only work on Mac OS X but I am sure there is way to achieve the same effect on other OSs.
(defn show []
(mash-up)
(.exec (Runtime/getRuntime) "open mash.html"))
Building a del.icio.us Link Recommender
16 Dec 2009
This is a port of the link recommendation engine described in the Programming Collective Intelligence book, to Clojure. In the book author used pydelicious API to build his dataset, I tried a bunch of Java APIs but they all sucked one way or another so I decided to use their public feeds instead of wasting time fighting with the APIs.
Let's get some includes out of the way,
(ns delicious
(:require [clojure.zip :as zip]
[clojure.xml :as xml])
(:use clojure.contrib.zip-filter.xml))
request function will take a URL to a RSS feed, parse it and return a zip structure,
(defn request [url]
(let [conn (-> (java.net.URL. url) .openConnection)]
(zip/xml-zip (xml/parse (.getInputStream conn)))))
To build the dataset, I retrieve the popular bookmarks for clojure and build a list of users to get bookmarks for,
(defn popular-users []
(let [feed (request "http://feeds.delicious.com/v2/rss/popular/clojure")]
(vec (xml-> feed :channel :item :dc:creator text))))
Then fetch bookmarks tagged Clojure for each user,
(defn user-bookmarks [user]
(let [feed (request
(str "http://feeds.delicious.com/v2/rss/" user "/clojure"))]
(reduce (fn[h v] (assoc h v 1) )
{} (xml-> feed :channel :item :link text))))
And build a map similar to the one used in Making Recommendations,
(defn preferences [users]
(reduce (fn[m v] (assoc m v (user-bookmarks v))) {} users))
The difference this preferences has from the previous one is that everyone in the map will have the same set of URLs, with a value of one if the user bookmarked it or a value of zero if user did not bookmarked it. We need to build a list of all the URLs in the preferences and add to each user URLs that they did not bookmarked with a value of zero,
(defn fill-in [prefs]
(let [all-items (reduce (fn[s v] (merge s v) ) {} (vals prefs))]
(reduce (fn[h v]
(let [user (first v)
prefs (second v)
diff (reduce (fn[h v]
(if (nil? (get prefs v))
(assoc h v 0) h)) {} (keys all-items))]
(assoc h user (merge prefs diff)))) {} prefs)))
Since building the preferences requires network I/O which takes time, I defined a variable to hold the preferences to save time while playing,
(def dic (fill-in (preferences (popular-users))))
As similarity score I choose to use Euclidean Distance.
(defn euclidean [person1 person2]
(let [shared-items (filter person1 (keys person2))
score (reduce (fn[scr mv]
(let [score1 (person1 mv)
score2 (person2 mv)]
(+ scr (Math/pow (- score1 score2) 2))))
0 shared-items)]
(if (= (count shared-items) 0)
0
(/ 1 (+ 1 score)))))
delicious=> (euclidean (dic "clojurebot") (dic "infrared"))
0.0625
delicious=> (euclidean (dic "clojurebot") (dic "clojurebot"))
1.0
Now we have everything setup to calculate how similar users are,
(defn similarities [prefs person algo]
(reduce
(fn[h p] (assoc h (first p) (algo (prefs person) (second p))))
{} (dissoc prefs person)))
delicious=> (similarities dic "clojurebot" euclidean)
{"snearch" 0.0625, "infrared" 0.0625, "rrc" 0.0625,
"atreyu_bbb" 0.0625, "precip" 0.0625, "pelleb" 0.07142857142857142,
"studiomaestro" 0.5, "mreid" 0.07142857142857142, "drcabana" 0.0625,
"jolby" 0.0625, "agriffin73" 0.0625}
Following four functions are copied and pasted from Making Recommendations, only one weight-prefs function has changed slightly since everyone has the same set of URLs, refer to that post for how they work,
(defn weight-prefs [prefs similarity person]
(reduce
(fn [h v]
(let [other (first v) score (second v)
diff (dic other)
weighted-pref (apply hash-map
(interleave (keys diff)
(map #(* % score) (vals diff))))]
(assoc h other weighted-pref))) {} similarity))
(defn sum-scrs [prefs]
(reduce (fn [h m] (merge-with #(+ %1 %2) h m)) {} (vals prefs)))
;(sum-scrs (weight-prefs dic (similarities dic "snearch" pearson) "snearch"))
(defn sum-sims [weighted-pref scores sim-users]
(reduce (fn [h m]
(let [movie (first m)
rated-users (reduce
(fn [h m] (if (contains? (val m) movie)
(conj h (key m)) h))
[] weighted-pref)
similarities (apply + (map #(sim-users %) rated-users))]
(assoc h movie similarities) ) ) {} scores))
(defn recommend [prefs person algo]
(let [similar-users (into {} (similarities prefs person algo))
weighted-prefs (weight-prefs prefs similar-users person)
scores (sum-scrs weighted-prefs)
sims (sum-sims weighted-prefs scores similar-users)]
(interleave (keys scores) (map #(/ (second %) (sims (first %))) scores))))
Now that everthing is set, lets get some recommendations,
delicious=> (take 5
(reverse
(sort-by second
(apply hash-map
(recommend dic "clojurebot" euclidean)))))
(["http://clojure.org/" 0.4375]
["http://intensivesystems.net/tutorials/cont_m.html" 0.2265624999]
["http://intensivesystems.net/tutorials/web_sessions.html" 0.226562]
["http://www.bestinclass.dk/index.php/2009/12...++Blog%29" 0.171875]
["http://www.tbray.org/ongoing/...ng-Clojure" 0.1640625])
Using MySQL with Clojure
14 Dec 2009
In a recent engagement, I needed to grab some data from a MySQL database. This write up is half cheat sheet, half tutorial for future reference. If you want to play with the snippets below create the following database and user,
$ mysql -u root -p
mysql> create database dummy;
Query OK, 1 row affected (0.00 sec)
mysql> grant all on dummy.* to 'duser'@'localhost' identified by 'dpass';
Query OK, 0 rows affected (0.08 sec)
Java uses an API called JDBC to access databases, each vendor provides drivers to access their database systems, MySQL uses Connector/J driver for access. Download the jar file and place it on your classpath.
clojure-contrib contains an interface to SQL databases via JDBC, first import the SQL interface,
(ns mysql
(:require [clojure.contrib.sql :as sql]))
SQL calls requires a map containing the connection properties,
(def db {:classname "com.mysql.jdbc.Driver"
:subprotocol "mysql"
:subname "//localhost:3306/dummy"
:user "duser"
:password "dpass"})
For creating and dropping databases, interface provides two functions,
(defn create-users []
(sql/create-table
:users
[:id :integer "PRIMARY KEY" "AUTO_INCREMENT"]
[:fname "varchar(25)"]
[:lname "varchar(25)"]))
(defn drop-users []
(sql/drop-table :users))
Calls are made using "with-connection" macro which takes the database properties we created and the function or functions we want to call,
(sql/with-connection db
(create-users))
Inserting data is accomplished via insert-values function,
(defn insert-user [fname lname]
(sql/insert-values :users [:fname :lname] [fname lname]))
(sql/with-connection db
(insert-user "Sandy" "Brown"))
Selecting data is done via "with-query-results" macro, which will return a sequence of maps,
(sql/with-connection db
(sql/with-query-results rs ["select * from users"]
(dorun (map #(println %) rs))))
mysql=> {:id 2, :fname Sandy, :lname Brown}
nil
To update a record,
(defn update-user [id attribute-map]
(sql/update-values :users ["id=?" id] attribute-map))
(sql/with-connection db
(update-user 1 {:fname "Sandy" :lname "Black"}))
To delete a record,
(defn delete-user [id]
(sql/with-connection db
(sql/delete-rows :users ["id=?" id])))
(sql/with-connection db
(delete-user 1))
For applications where SQL queries are constructed from user input, prepared statements should be used instead to prevent against SQL Injection attacks,
(let [sql "insert into dummy.users (fname,lname) values (? , ?)"]
(sql/with-connection db
(sql/do-prepared sql ["Sandy" "Brown"] )))
(sql/with-connection db
(sql/with-query-results rs ["select * from users where id=?" 3]
(dorun (map #(println %) rs))))
Getting etags to Index Clojure Files
13 Dec 2009
etags is a great way to move around in a project, it's fast and doesn't get in your way, but out of the box etags does not recognize clojure files even with the
--language=lisp
option, it won't index correctly. Fortunately etags does support an option to take in a file containing regular expressions to recognize unknown languages, save the following regular expressions in a file,
/[ \t\(]*def[a-z]* \([a-z-!]+\)/\1/
/[ \t\(]*ns \([a-z.]+\)/\1/
Then running the following command will get etags to recognize and index clojure files.
find . -name '*.clj' | xargs etags --regex=@/path/to/tags.file
Creating Mac OS X App Bundle for Java Applications
12 Dec 2009
Java support in Mac OS X isn't perfect but it is better than other operating systems, Apple supplies an application called Jar Bundler, that can wrap your jar files into native Mac OS X App bundles, that are pretty much indistinguishable from native applications.
As of this writing location of the Jar Bundler is,
open /usr/share/java/Tools/Jar\ Bundler.app/
they changed the location couple of times, so use Spot Light to locate it.
Jar Bundler is pretty straight forward to use, on "Build Information" panel choose your jar, set any arguments you wish to pass to your jar. If your application depends on third party Jars, you can add them to the bundle in the "Classpath and Files" tab. When you click "Create Application", Jar Bundler will create a double clickable Mac OS X App bundle for your application.
Of course, it is not practical to manually create application bundles with every build, but once you create an application bundle you can integrate it into your build process, application bundle is a folder, navigate into it then copy the following files and folders,
- Info.plist
- MacOS/
- PkgInfo
- Resources/
into a directory under your source tree, for the following examples I assume they are in a folder called "macApp". Now we can create an ant task to recreate the bundle directory structure after the build.
<target name="app" depends="">
<mkdir dir="${build.dir}/Your.app" />
<mkdir dir="${build.dir}/Your.app/Contents/" />
<copy todir="${build.dir}/Your.app/Contents/">
<fileset dir="macApp/"/>
</copy>
<mkdir dir="${build.dir}/Your.app/Contents/Resources/Java/" />
<copy file="${build.dir}/your.jar"
todir="${build.dir}/Your.app/Contents/Resources/Java/"/>
<chmod file="${build.dir}/Your.app/Contents/MacOS/JavaApplicationStub"
perm="700"/>
</target>
Mac applications are distributed in Apple Disk Image files, to build a .dmg image for our application we add another ant task,
<target name="distDarwin" depends="app">
<exec dir="${build.dir}" executable="hdiutil">
<arg line="create -ov -srcfolder Your.app Your.dmg"/>
</exec>
<exec dir="${build.dir}" executable="hdiutil">
<arg line="internet-enable -yes Your.dmg"/>
</exec>
</target>
With these tasks, you can integrate application bundling to your build process and have a distributable image with a single command.
Type Less to Type More
11 Dec 2009
Emacs has two modes,
when combined, these modes allow you to type half as much and produce twice as much code. An abbrev is a string of characters, that will be expanded to a longer string, skeletons on the other hand are templates written in a mini language, implemented in elisp. When an abbrev is set to expand into a skeleton, you can write chunks of code with just a few keystrokes.
One of the most frequent statements I use for debugging is the print call, instead of typing,
System.out.println(" some thing.. " );
every time, I can just type "prt" and when I hit space it will be expanded to the form above. For this we need to define a skeleton for it,
(define-skeleton skel-java-println
"Insert a Java println Statement"
nil
"System.out.println(" _ " );")
"_" denotes where the cursor will be placed after the expansion. Now to test this use M-x "skel-java-println"
System.out.println( );
it should insert the code and place the cursor inside the parenthesis. Using M-x is nice but it is still way to slow for our purposes, next we define a abbrev for java-mode,
(define-abbrev java-mode-abbrev-table "prt" "" 'skel-java-println)
Now every time we type "prt" in java-mode, it will be expanded to a print statement. Of course you are not limited to one liners,
(define-skeleton skel-java-try
"Insert a try catch block"
nil
\n >
"try{"
\n >
_ \n
"}catch( Exception e ) {" >
" "
\n > \n
"}" >)
will expand to,
try{
}catch( Exception e ) {
}
here "\n" denotes a new line and ">" means the line will be indented, to match where you are in the code.
To activate abbrev-mode add,
(setq abbrev-mode t)
to your .emacs file, along with the skeletons. Now Emacs may complain as you load your .emacs file that some modes doesn't have abbrev tables, in that case define them before the skeleton definitions,
(define-abbrev-table 'java-mode-abbrev-table '())
If the mode doesn't have any abbrev table set, you also need to set the table for use in the mode this happened to me with clojure-mode,
(add-hook 'clojure-mode-hook
(lambda ()
(setq local-abbrev-table clojure-mode-abbrev-table)))
While at it, enable skeleton-pair mode,
(setq skeleton-pair t)
(global-set-key (kbd "(") 'skeleton-pair-insert-maybe)
(global-set-key (kbd "[") 'skeleton-pair-insert-maybe)
(global-set-key (kbd "{") 'skeleton-pair-insert-maybe)
(global-set-key (kbd "\"") 'skeleton-pair-insert-maybe)
now every time you type [ ( " { , these will be inserted in pairs, one less thing to type.
Poor Man's Arduino Line Follower
09 Dec 2009
For a faster, smarter line follower checkout Arduino Line Follower Take Two
Over the weekend, me and a friend of mine, cannibalized a 9 Lira (6$) RC car into line a follower. Code in this post covers everything up to commit cf731aedc156f067aa221, refer to this particular commit for this project cause we plan on improving hence code will change.
Stuff you will need,
- 3x QTR-1RC Reflectance Sensor
- Ardumoto - Motor Driver Shield
- A cheap RC car
Hardware
Wiring looks like the following,
Fritzing project can be downloaded here. Sorry about the crappy diagram, but I just suck at it.
Two sensors track the black area to the sides, one sensor tracks the white line in the middle. Each sensor has 3 pins on them, VIN GROUND and OUT. VIN and GROUND goes to +5V and GROUND, OUT goes to digital pin. For this project we used, digital pins 5 through 7.
- Sensor on the left is connected to digital pin 5
- Sensor in the middle is connected to digital pin 6
- Sensor on the right is connected to digital pin 7
An LED is connected to digital pin 8, and a push button is connected to digital pin 2, refer to the fritzing diagram for their wiring.
To control the DC motors on the RC car, we used an ardumoto shield which allows you to control up to 2 DC motors. It uses digital pins,
- 10 for PWM for MotorA
- 11 for PWM for MotorB
- 12 for Direction MotorA
- 13 for Direction MotorB
This particular car is setup such that the motor on the rear provides forward and backward motion depending on the direction of the motor, forward motor turns the wheels depending on the direction it is turning (e.g. when the direction pin is high wheels turn right).
Power supplied through a 9V 2Amp power adapter.
Software
Code is divided into 2 modules, engine and navigation. Engine module is responsible for movement it exposes five functions, straight, left, right, reverse and forward. Navigation module exposes 2 functions calibrate and steer.
void engine::forward(int speed, int time){
analogWrite(PwmPinMotorB, speed);
digitalWrite(DirectionPinMotorB, LOW);
delay(time);
analogWrite(PwmPinMotorB, 0);
digitalWrite(DirectionPinMotorB, LOW);
}
void engine::reverse(int speed, int time){
analogWrite(PwmPinMotorB, speed);
digitalWrite(DirectionPinMotorB, HIGH);
delay(time);
analogWrite(PwmPinMotorB, 0);
digitalWrite(DirectionPinMotorB, HIGH);
}
These functions can be used to move the robot forward and backward. They take a PWM value and a time in milliseconds,
engine.forward(255,10);
will turn the rear motor for 10 milliseconds at full power. Direction of the robot can be changed using,
void engine::right(){
analogWrite(PwmPinMotorA, 255);
digitalWrite(DirectionPinMotorA, LOW);
}
void engine::left(){
analogWrite(PwmPinMotorA, 255);
digitalWrite(DirectionPinMotorA, HIGH);
}
void engine::straight(){
analogWrite(PwmPinMotorA, 0);
digitalWrite(DirectionPinMotorA, HIGH);
}
These will change the direction of the forward motor causing the car to turn. Turning the motor forward will cause the front wheel to turn left, turning the motor backwards causes the wheels to turn right.
Navigation module begins, by calibrating itself,
void navigation::calibrate(){
unsigned int val[3];
qtr.read(val);
int right = val[0];
int middle = val[1];
int left = val[2];
bwMean = ((right - middle) + (left - middle))/2;
}
We read all three sensors and calculate the average of difference in black and white readings, which is stored in a variable called bwMean, it is used when calculating robots position relative to the line.
void navigation::steer(){
int bearing = getBearing();
if(bearing == LEFT)
engin.left();
if(bearing == STRAIGHT)
engin.straight();
if(bearing == RIGHT)
engin.right();
engin.forward(255,10);
}
We ask for a bearing then tell the engine to turn in that direction and move forward one step.
int navigation::getBearing(){
unsigned int val[3];
qtr.read(val);
int right = val[0];
int middle = val[1];
int left = val[2];
if(left <= middle && left <= right && position != RIGHT){
position = LEFT;
return RIGHT;
}
if(right <= left && right <= middle && position != LEFT){
position = RIGHT;
return LEFT;
}
if(middle <= left && middle <= right && middle < bwMean){
position = STRAIGHT;
return STRAIGHT;
}
if (position == LEFT ) return RIGHT;
if (position == RIGHT ) return LEFT;
}
getBearing calculates our position relative to the line. We read all three sensors, basically the sensor which has the lowest value is on the white line. So we turn in the direction of the sensor with the lowest value, but there is a problem with this approach, when all sensors are on the black area, one of them will still read lower than the others which will cause the robot to turn in some random direction. To overcome this problem we introduce a new variable called position which records last position. Now we only turn right or left if we are not already in that side of the line, e.g. we don't want to turn left when we are already left of the line. This covers left and right turns however when all the sensors are on the black, middle one will still read lowest so we use the bwMean value we calculated and only go straight if the middle sensor reading is lower than the mean value we calculated. If we are left or right of the line we continue turning in that direction, until we find the line again.
In the main arduino loop we just call steer function over and over again,
void loop(){
navigation.steer();
delay(70);
}
When everything wired and code uploaded, it works like this,
For more pictures and videos check out my flickr set.
iTunes Navigation Header
08 Dec 2009
If you are one of those people, who thinks swing is ugly, I suggest checking out, Exploding Pixels, blog of Ken Orr creator of macwidgets.
Two of his recent posts,
covers the process of recreating iTunes navigation header (seen in the iTunes music store - the black shiny bar at the top). I wanted to try it on a toy project of mine, following is a direct translation of his code to Clojure, for a detailed explanation of the code check his posts.
(defn itunes-header [layout]
(let [header-height 25
;the background colors used in the multi-stop gradient.
background-color-1 (java.awt.Color. 0x393939)
background-color-2 (java.awt.Color. 0x2e2e2e)
background-color-3 (java.awt.Color. 0x232323)
background-color-4 (java.awt.Color. 0x282828)
;the color to use for the top and bottom border.
border-color (java.awt.Color. 0x171717)
;the inner shadow colors on the top of the header.
top-shadow-color-1 (java.awt.Color. 0x292929)
top-shadow-color-2 (java.awt.Color. 0x353535)
top-shadow-color-3 (java.awt.Color. 0x383838)
;the inner shadow colors on the bottom of the header.
bottom-shadow-color-1 (java.awt.Color. 0x2c2c2c)
bottom-shadow-color-2 (java.awt.Color. 0x363636)]
(proxy [javax.swing.JPanel] [layout]
(getPreferredSize [] (java.awt.Dimension. -1 header-height))
(paintComponent
[g]
(let [graphics (cast java.awt.Graphics2D (.create g))
height (.getHeight this)
width (.getWidth this)
mid-y (int (/ height 2))]
(doto graphics
;paint the top half of the background with
;the corresponding gradient
(.setPaint (java.awt.GradientPaint. 0 0 background-color-1
0 mid-y background-color-2))
(.fillRect 0 0 width mid-y)
;paint the top half of the background with
;the corresponding gradient
(.setPaint
(java.awt.GradientPaint. 0 (+ mid-y 1) background-color-3
0 height background-color-4))
(.fillRect 0 mid-y width height)
;draw the top inner shadow
(.setColor top-shadow-color-1)
(.drawLine 0 1 width 1)
(.setColor top-shadow-color-2)
(.drawLine 0 2 width 2)
(.setColor top-shadow-color-3)
(.drawLine 0 3 width 3)
;draw the bottom inner shadow.
(.setColor bottom-shadow-color-1)
(.drawLine 0 (- height 3) width (- height 3))
(.setColor bottom-shadow-color-2)
(.drawLine 0 (- height 2) width (- height 2))
;draw the top and bottom border
(.setColor border-color)
(.drawLine 0 0 width 0)
(.drawLine 0 (- height 1) width (- height 1)))
(.dispose graphics))))))
(defn itunes-header-button-ui []
(let [text-color java.awt.Color/WHITE
text-shadow-color java.awt.Color/BLACK
;the gradient colors for when the button is selected.
selected-background-color-1 (java.awt.Color. 0x141414)
selected-background-color-2 (java.awt.Color. 0x1e1e1e)
selected-background-color-3 (java.awt.Color. 0x191919)
selected-background-color-4 (java.awt.Color. 0x1e1e1e)
;the border colors for the button.
selected-top-border (java.awt.Color. 0x030303)
selected-bottom-border (java.awt.Color. 0x292929)
;the border colors between buttons.
left-border (java.awt.Color. 255,255,255,21)
right-border (java.awt.Color. 0,0,0,125)
selected-inner-shadow-color-1 (java.awt.Color. 0x161616)
selected-inner-shadow-color-2 (java.awt.Color. 0x171717)
selected-inner-shadow-color-3 (java.awt.Color. 0x191919)]
(proxy [javax.swing.plaf.basic.BasicButtonUI] []
(installDefaults
[button]
(proxy-super installDefaults button)
(.setBackground button (java.awt.Color. 0 0 0 0))
(.setOpaque button false))
;if the button is selected, paint the special background now.
;if it is not selected paint the left and right highlight border.
(paint
[graphics component]
(let [button (cast javax.swing.AbstractButton component)]
(if (= (.isSelected button) true)
(.paintButtonPressed this graphics button)
(do
(doto graphics
(.setColor left-border)
(.drawLine 0 1 0 (- (.getHeight button) 2))
(.setColor right-border)
(.drawLine
(- (.getWidth button) 1) 1
(- (.getWidth button) 1) (- (.getHeight button) 2)))))
(proxy-super paint graphics component)))
; we need to override the paintText method so that we can paint
; the text shadow. the paintText method in BasicButtonUI pulls
; the color to use from the foreground property -- there is no
; way to change this during the painting process without causing
; an infinite sequence of events, so we must implement our own
; text painting.
(paintText
[graphics button text-rect text]
(let [font-metrics (.getFontMetrics graphics (.getFont button))
mnemonix-index (.getDisplayedMnemonicIndex button)]
;paint the shadow text.
(.setColor graphics text-shadow-color)
(javax.swing.plaf.basic.BasicGraphicsUtils/drawStringUnderlineCharAt
graphics text mnemonix-index
(+ (.x text-rect) (.getTextShiftOffset this))
(+ (.y text-rect)
(.getAscent font-metrics)
(.getTextShiftOffset this) -1))
;paint the actual text.
(.setColor graphics text-color)
(javax.swing.plaf.basic.BasicGraphicsUtils/drawStringUnderlineCharAt
graphics text mnemonix-index
(+ (.x text-rect) (.getTextShiftOffset this))
(+ (.y text-rect)
(.getAscent font-metrics)
(.getTextShiftOffset this)))))
;Paints the selected buttons state, also used as the pressed state.
(paintButtonPressed
[g button]
(let [height (.getHeight button)
width (.getWidth button)
mid-y (int (/ height 2))
graphics (cast java.awt.Graphics2D g)]
(doto graphics
(.setPaint
(java.awt.GradientPaint. 0 0 selected-background-color-1
0 mid-y selected-background-color-2))
(.fillRect 0 0 width mid-y)
(.setPaint
(java.awt.GradientPaint.
0 (+ 1 mid-y) selected-background-color-3
0 height selected-background-color-4))
(.fillRect 0 mid-y width height)
;draw the top and bottom border.
(.setColor selected-top-border)
(.drawLine 0 0 width 0)
(.setColor selected-bottom-border)
(.drawLine 0 (- height 1) width (- height 1))
;paint the outter part of the inner shadow.
(.setColor selected-inner-shadow-color-1)
(.drawLine 0 1 0 (- height 2))
(.drawLine 0 1 width 1)
(.drawLine (- width 1) 1 (- width 1) (- height 2))
;paint the middle part of the inner shadow.
(.setColor selected-inner-shadow-color-2)
(.drawLine 1 1 1 (- height 2))
(.drawLine 0 2 width 2)
(.drawLine (- width 2) 1 (- width 2) (- height 2))
;paint the inner part of the inner shadow.
(.setColor selected-inner-shadow-color-3)
(.drawLine 2 1 2 (- height 2))
(.drawLine 0 3 width 3)
(.drawLine (- width 3) 1 (- width 3) (- height 2))))))))
Glue them together,
(let [frame (javax.swing.JFrame. "This is a test")
panel (javax.swing.JPanel.
(net.miginfocom.swing.MigLayout. "fillx" ""))
header (itunes-header
(net.miginfocom.swing.MigLayout. "insets 0 0 0 0, gapx 0"))
music (javax.swing.JButton. "Music")
movies (javax.swing.JButton. "Movies")]
(.setUI music (itunes-header-button-ui))
(.setUI movies (itunes-header-button-ui))
(.add header music )
(.add header movies)
(.add panel header "north")
(doto frame
(.add panel)
(.setSize 400 150)
(.setVisible true)))
Result is a very nice looking iTunes like header,
Zipping XML with Clojure
07 Dec 2009
I've written about parsing XML with Clojure before, the problem is, as I wanted more from the document, loops started to get uglier. I thought it was time to try the zip-filter API, it provides XPath style navigation in Clojure. Not much documentation about it exists but as usual source code was very helpful.
We begin by including the required libraries,
(ns zp
(:require [clojure.zip :as zip]
[clojure.xml :as xml])
(:use clojure.contrib.zip-filter.xml))
And a utility function to convert a XML string into a zip structure,
(defn zip-str [s]
(zip/xml-zip (xml/parse (java.io.ByteArrayInputStream. (.getBytes s)))))
I was playing with the XML example snippet from Wikipedia, which it claims has all the XML concepts in it.
(def paintings (zip-str "<?xml version='1.0' encoding='UTF-8'?>
<painting>
<img src='madonna.jpg' alt='Foligno Madonna, by Raphael'/>
<caption>This is Raphael's 'Foligno' Madonna, painted in
<date>1511</date>-<date>1512</date>.</caption>
</painting>"))
xml-> call provides a way to navigate through the elements and attributes in an XML document. If we want to access the caption tag we can use,
(xml-> paintings :caption text)
Or to get the dates it was painted in,
(xml-> paintings :caption :date text)
For accessing attributes, there is the attr function which returns the matching XML attribute,
(xml-> paintings :img (attr :src))
These calls return a sequence, you can either call first on them or use the xml1-> function, which calls it for you. zip-filter can do more than these simple examples, these are what I needed at the moment for more examples, have a look at the bottom of the source code.
Java Taskbar Icon on Windows 7
06 Dec 2009
I was testing an application, on Windows 7 which had a 25x25 png image to be used as the taskbar icon. Windows 7's new taskbar uses much bigger icons so my 25x25 png turned in to big blur. There is a solution however, Java 1.6 introduces a new function called, setIconImages.
public void setIconImages(List<? extends Image> icons)
Build a list of images of different sizes, set it with setIconImages, instead of setIconImage, and Windows will pick the right size depending on the context (e.g. window decoration, window list, taskbar, etc.).
In my case I needed compatibility with Java 1.5, so I ended up calling setIconImage via reflection and if an exception is raised I fall back to old setIconImage function.
try{
Class[] types = {java.util.List.class};
java.lang.reflect.Method method =
Class.forName("java.awt.Window")
.getDeclaredMethod("setIconImages",types);
Object[] params = {images};
method.invoke(frame,params);
}catch( Exception e ) {
frame.setIconImage((Image)images.get(0));
}
Distributed Clojure Using RMI
05 Dec 2009
The Java Remote Method Invocation API, or RMI is the Java way of doing remote procedure calls (RPC). Remote Method Invocation (RMI) facilitates object function calls between Java Virtual Machines (JVMs). JVMs can be located on separate computers, one JVM can invoke methods belonging to an object stored in another JVM. I have been meaning to play with RMI from Clojure for a while, so I have put together a distributed "Hello, World!" application.
A RMI application is made up of three parts,
- Hello.java - Remote Interface
- server.clj - Server
- client.clj - Client
Remote Interface
Remote interface contains the methods that are exposed by the server that can be called from another JVM, and it is the only part of code that can't be written in Clojure, every method in the interface must throw RemoteException, this can't be done with gen-interface macro. If you create your interfaces with gen-interface you will get exceptions.
package stub;
import java.rmi.Remote;
import java.rmi.RemoteException;
public interface Hello extends Remote {
String sayHello() throws RemoteException;
}
Our interface exposes a single function called sayHello, when called, it will return the "Hello, World!" string. Compile this file,
javac Hello.java
and place it in a folder called stub, place this stub folder anywhere on your classpath.
Server
First thing, server has to do is to start remote object registry, servers and clients locate each other using this registry mechanism.
(def rmi-registry (java.rmi.registry.LocateRegistry/createRegistry 1099))
We start the RMI registry on port 1099, if we need to programmatically stop the registry for any reason we can use the following function,
(defn stop-rmi []
(java.rmi.server.UnicastRemoteObject/unexportObject rmi-registry true))
We need to create an instance of the interface we defined, containing the implementations of the functions we exposed in the interface,
(defn hello-server []
(proxy [stub.Hello] []
(sayHello [] "Hello, World!") ))
Server needs to be registered with the RMI registry before it can be called from another JVM.
(defn register-server []
(.bind
(java.rmi.registry.LocateRegistry/getRegistry)
"Hello"
(java.rmi.server.UnicastRemoteObject/exportObject (hello-server) 0)))
We bind the name "Hello" to our hello-server, now server is ready to accept connections, we register the server and wait for incoming connections.
(register-server)
(while true (Thread/sleep 1000))
Client
On the client side, first we locate the registry,
(def rmi-registry (java.rmi.registry.LocateRegistry/getRegistry "127.0.0.1"))
Replace 127.0.0.1 with the IP of the server machine if the server is on another IP. Now that we have located the registry, we can lookup objects by name.
(let [hello (.lookup rmi-registry "Hello")]
(println (.sayHello hello)))
Thats all the code that is needed for a distributed "Hello, World!". To put it all to action, run server.clj, it will start listening for incoming connections, on the same machine or on another machine on the same network, run client.clj, you should get "Hello, World!" returned to the client.
BitTorrent Tracker Protocol
03 Dec 2009
Yesterday I got some free time and I thought I would complete another piece from BitTorrent protocol. In order to download a torrent, you need to communicate with the tracker and get a list of computers that are also downloading the torrent you are interested in.
Tracker is an HTTP service that responds to GET requests. Our request includes statistics about us that helps the tracker keep metrics about the torrent. The response from the tracker includes a peer list and some basic statistics about the torrent. Specs are at theory.org and I used torrents from ubuntu distribution for the tests.
If you want to jump to the code tracker.clj is here bencode.clj is here in bencode.clj, two functions needs to be replaced, to get correct SHA1 hashes. Replace the function below with the ones in bencode.clj.
(defn- decode-map [stream]
(let [list (decode-list stream)]
(with-meta
(apply hash-map list)
{:order (map first (partition 2 list))})))
(defn- encode-dictionary [dictionary stream]
(.write stream (int \d))
(doseq [item (if (nil? (meta dictionary))
(keys dictionary)(:order (meta dictionary)))]
(encode-object item stream)
(encode-object (dictionary item) stream))
(.write stream (int \e)))
The only difference with this one and the original is we keep the order keys are read as meta data. The during encoding we encode according to the order we read them, that way we get the original torrent file back which will have a correct SHA1 hash. If you don't do this, map iteration order will be different than the original and different hash will be calculated.
Most important part of the request is the info hash part which identifies which torrent we are interested in. It is the SHA1 hash of the info dictionary not the whole file, in its bencoded form. That is the reason we need to keep the order, same as the torrent file, if you mess up the order hash will change.
(defn calc-info-hash [torrent]
(let [info (encode (torrent "info"))
sha1 (MessageDigest/getInstance "SHA1")
hash (.toString (BigInteger. (.digest sha1 info)) 16)
pad (- 40 (count hash))]
(str (apply str (take pad (repeat "0"))) hash)))
We are encoding the info dictionary using original order, then calculate the SHA1 hash. SHA1 hash is a byte array to turn it in to hex, I used the BigInteger class, there are lots of ways to do this I went with the simplest. Pad the begging with 0s if it is less then 40 chars long. Result is a hash such as the following,
3e427a0a9d4826e76cd5363a004b2fa6baca1853
If you don't pay attention to the spec and send this directly to tracker you will get an error this should be in URL Encoded form. Padding every two chars with % sign also doesn't work, been there done that don't waste your time. Any hex in the hash that corresponds to a unreserved character should be replaced,
a-z A-Z 0-9 -_.~
Partition the hex in to chunks of two and check if the hex corresponds to any of these values, if they do replace them with the unreserved char,
(defn url-encode [hash]
(apply str
(map (fn [[a b]]
(let [byte (BigInteger. (str a b) 16) ]
(if (or (and (>= byte 65) (<= byte 90)) ; A-Z
(and (>= byte 97) (<= byte 122)) ; a-z
(and (>= byte 48) (<= byte 57)) ; 0-9
(= byte 45) (= byte 95) (= byte 46) (= byte 126))
(char byte) (str "%" a b)) )) (partition 2 hash))))
So that a hash such as,
123456789abcdef123456789abcdef123456789a
becomes,
%124Vx%9a%bc%de%f1%23Eg%89%ab%cd%ef%124Vx%9a
notice that hex 34 became 4 which is what it is in ASCII. You can test the correctness of your hashes using the tracker url but don't request from announce request from file,
http://some.tracker.com/file?info_hash=hash
If you get a torrent back that means you have the correct hash.
Requests to announce requires all parameters to be set in order to be valid, you can't build the request one by one, a working request should include all of the following parameters.
(defn build-request [torrent]
(let [announce (torrent "announce")
hash (calc-info-hash torrent)
event "started"]
(str announce "?"
"info_hash=" (url-encode hash) "&"
"peer_id=" peer-id "&" "port=" port-in "&"
"uploaded=0&downloaded=0&" "left=" ((torrent "info") "length") "&"
"event=" event "&" "numwant=" peer-list-size "&compact=1" )))
Parameters are pretty much self explanatory, they are also explained in the spec.
Tracker response is a bencoded dictionary containing statistics and peers list. Peers value in the response map is a byte string multiple of 6, every 6 bytes represent a peer. First 4 bytes contains the IP and last two bytes contains the port they are listening on.
(defn peers [peers]
(reduce (fn[list peer]
(conj list
{:ip (apply str (interpose \. (map int (take 4 peer))))
:port (+ (* 256 (int (nth peer 5))) (int (nth peer 4)))}))
[] (partition 6 peers)))
To parse this byte string we partition it in to 6 byte chunks, map first 4 bytes to a integer to build the IP address, and use last two bytes to build port number.
Now we have everything to extract some basic information from the tracker,
(defn get-torrent-stats [fname]
(let [torrent (decode (FileInputStream. (File. fname)))
request (request (build-request torrent))
stats (decode (ByteArrayInputStream. (.getBytes request)))]
{:complete (stats "complete")
:incomplete (stats "complete")
:peers (peers (stats "peers"))} ))
Putting it all together, and making the request should result it a map containing the peers and statistics.
tracker=> (get-torrent-stats "buntu.torrent")
{:complete 0, :incomplete 0,
:peers [{:ip "84.29.195.139", :port 40898}
{:ip "195.153.89.87", :port 49768}]}
Adding Inferior Lisp Support for clojure-mode
01 Dec 2009
It has been a while since I updated clojure-mode, after the update inferior-lisp stopped working. At first I thought I messed up something because I updated clojure, clojure-contrib, clojure-mode all at the same time (bad move!). After making sure clojure and clojure-contrib works I looked at the clojure-mode's source code and found this commit which removes the subprocess support (inferior-lisp) for clojure-mode and advises to use slime and swank-clojure. Slime is nice but I wanted my inferior-lisp support back.
If you don't have any setup for inferior-lisp support use the following snippet, setting the classpath according to your needs,
(setq class-path (concat "-cp "
"./extLibs/*:"
"../extLibs/*:"
"./classes/:"
"."))
(setq clojure-command (concat "java -server -Dfile.encoding=UTF-8 "
class-path " clojure.lang.Repl" ))
(setq inferior-lisp-program clojure-command)
This will let inferior-lisp to run clojure REPL, to be able to send code from your clojure buffer to inferior-lisp buffer use the following key bindings.
(defun na-load-buffer ()
(interactive)
(point-to-register 5)
(mark-whole-buffer)
(lisp-eval-region (point) (mark) nil)
(jump-to-register 5))
(add-hook 'clojure-mode-hook
'(lambda ()
(define-key clojure-mode-map
"\e\C-x" 'lisp-eval-defun)
(define-key clojure-mode-map
"\C-x\C-e" 'lisp-eval-last-sexp)
(define-key clojure-mode-map
"\C-c\C-e" 'lisp-eval-last-sexp)
(define-key clojure-mode-map
"\C-c\C-r" 'lisp-eval-region)
(define-key clojure-mode-map
"\C-c\C-l" 'na-load-buffer)
(define-key clojure-mode-map
"\C-c\C-z" 'run-lisp)))
From the commit it seems that the only thing that got removed is bunch of key bindings, snippet above pretty much reverses the commit and adds a new function to send the buffer you are working on to REPL and gives you a functional inferior-lisp REPL.