Planet Scheme

Jeremy Kun: Cryptanalysis with N-Grams

2012-02-03T19:52:16+00:00

This post is the third post in a series on computing with natural language data sets. For the first two posts, see the relevant section of our main content page. A Childish Bit of Fun In this post, we focus … Continue reading →

Programming Praxis: Roman Numeral Puzzle

2012-02-03T09:00:27+00:00

The Super Bowl is the championship game of the American league of professional football teams — and an annual cultural event. The first Super Bowl, won by the Green Bay Packers under head coach Vince Lombardi, was in 1967, and wasn’t even called “Super Bowl” at the time; the name didn’t attach to the game until Super Bowl III, when Joe Namath of the New York Jets guaranteed a victory. Roman numerals have been used to designate the Super Bowl ever since. The game this year, between the Boston Patriots and New York Giants, is Super Bowl XLVI; you can solve today’s exercise while you watch the game on Sunday evening.

John D. Cook turned the wacky roman numerals of the Super Bowl into an exercise at his blog; he was inspired by an advertisement for Super Bowl XLVI on a pizza box. Cook asked his readers to compute how many numbers can be expressed as roman numerals without duplicating any of the symbols I, V, X, L, C, D or M.

Your task is to compute the number of roman numerals that have no duplicate characters. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

Jeremy Kun: The Fundamental Theorem of Algebra (with Galois Theory)

2012-02-03T04:42:11+00:00

This post assumes familiarity with some basic concepts in abstract algebra, specifically the terminology of field extensions, and the classical results in Galois theory and group theory. The fundamental theorem of algebra has quite a few number of proofs (enough … Continue reading →

PLT Scheme: Racket v5.2.1

2012-02-02T20:30:51+00:00

Racket version 5.2.1 is now available from

http://racket-lang.org/

Release Highlights:

Performance improvements include the use of epoll()/kqueue() instead of select() for the Racket thread scheduler, cross-module inlining of small functions, and the use of SSE instead of x87 for JIT-compiled floating-point operations on platforms where SSE is always available (including x86_64 platforms). A related change is the interning of literal numbers, strings, byte strings, characters, and regexps that appear in code and syntax objects.
DrRacket uses a set of composable ray-traced icons available from the new images library collection.
Typed Racket's typecheck-fail form allows macro creators to customize the error messages that Typed Racket produces. This is especially useful when creating pattern matching macros.
The performance of Redex's matcher has been substantially improved; depending on the model you should see improvements between 2x and 50x in the time it takes to reduce terms.
Plots look nicer and are more correct at very small and very large scales. New features include customizable dual axis ticks and transforms (e.g., log axes, date and currency ticks, axis interval collapse and stretch), stacked histograms, and 3D vector fields. The legacy fit function and libfit have been removed.
The 2htdp/universe library's big-bang form supports an experimental game pad key handler.
The db library now supports nested transactions and PostgreSQL arrays. Bugs involving MySQL authentication and memory corruption in the SQLite bindings have been fixed.
The Macro Stepper tool in DrRacket no longer executes a program after expanding it.
In the DMdA teaching languages, infinite recursive signatures ("streams", for example) with no intervening mixed are now supported, and the signatures of record definitions without fields now have generators for use with property.
MysterX's ActiveX support is deprecated and will be removed in the next release. MysterX's core COM functionality will become deprecated in the next release, but COM functionality will be supported for the foreseeable future as a compatibility layer over a forthcoming ffi/com library.

PLT Scheme: Zack Galler's Experience with Stateful vs Stateless Web Apps

2012-02-02T02:46:59+00:00

Communication using HTTP between client and server is a simple problem of halted computation.

A client computes a request, transmits and halts, waiting for a server response. On receipt, the server computes a response, transmits and halts, waiting for the next client request.

This much is well known.

Racket's magnificent stateful Web server does three things on the server side:

it reifies a Racket continuation, capturing where the server computation has halted.
it externalizes the continuation, creating a URL-representation that uniquely maps to the Racket continuation
it disseminates the externalized continuation to interested clients, typically via HTTP response, but alternately via SMTP or any other protocol.

Then, it waits.

Later, when presented with an externalized continuation, a quick inverse mapping occurs, the underlying Racket continuation is invoked, and the server processes the new client request.

Rinse and repeat.

The problem with this approach is twofold

the reified Racket continuations live in server memory. And there's no safe way to garbage collect, as the continuations could be invoked at any time. There are strategies to reclaim memory, but some load level will noticeably decrease the performance of your application. And its not possible to figure out what that load level is prior to finishing your application. This is a problem.
Again, the reified Racket continuations live in server memory and cannot be moved. So there's no way to scale an application to more than one server. It's a necessarily one machine system. This makes problem #1 worse.

Racket's yet more magnificent stateless Web server does exactly the same three things:

to reify, it rewrites the entire call stack into a format known as A-Normal Form (ANF).
to externalize, the ANF'd stack is encoded for transmission over HTTP.
and then it's sent over to the client (dissemination).

Later, when presented with encoded stack, the stateless server performs an inverse transform to reconstruct the call stack, at which point the server keeps going.

So we've lost the invocation step and substituted a reconstruction.

But in exchange, we've eliminated continuations from server memory, and solved both enumerated problems above. Neat trick.

I provide a few lessons learned for the archives for the next person to attempt porting #lang racket to #lang web-server code.

First, the predicate serializable? from racket/serialize is invaluable. The #lang web-server code will not transform if there are non-serializable constructs in the dynamic extent of the invocation of send/suspend, such as a local binding or argument.

Second, invocations of native continuations reified with call/cc frequently throw errors related to continuation prompts, such as “attempt to cross a continuation barrier” or “no corresponding prompt tag in continuation”. In all cases, I was able to remedy the situation by enclosing the invocation in call-with-continuation-prompt. This may be an error in the system, but it is unclear at this time.

Third, the transformation does not allow parameters or dynamic-wind, because the internal data-structures representing them are not serializable, but continuation-marks can be used to reimplement the piece of the functionality you need.

Finally, thank you to the Racket team. I think the stateless Web language is important technology and must have required an enormous amount of work to implement.

Anecdotally, application speed seems at or better than the stateful code.

To learn more about the stateless Web application infrastructure, consult the manual or post to the mailing list.

(This post was written by Zack Galler with minor edits before posting by Jay McCarthy.)

Andy Wingo: eval, that spectral hound

2012-02-01T15:33:49+00:00

Friends, I am not a free man. Eval has been my companion of late, a hellhound on my hack-trail. I give you two instances.

the howl of the-environment, across the ages

As legend has it, in the olden days, Aubrey Jaffer, the duke of SCM, introduced low-level FEXPR-like macros into his Scheme implementation. These allowed users to capture the lexical environment:

(define the-environment
  (procedure->syntax
   (lambda (exp env)
     env)))

Tom Lord inherited this cursed bequest from Jaffer, when he established himself in the nearby earldom of Guile. It so affected him that he added local-eval to Guile, allowing the user to evaluate an expression within a captured local environment:

(define env (let ((x 10)) (the-environment)))
(local-eval 'x env)
=> 10
(local-eval '(set! x 42) env)
(local-eval 'x env)
=> 42

Since then, the tenants of the earldom of Guile have been haunted by this strange leakage of the state of the interpreter into the semantics of Guile.

When the Guile co-maintainer title devolved upon me, I had a plan to vanquish the hound: to compile Guile into fast bytecode. There would be no inefficient association-lists of bindings at run-time. Indeed, there would be no "environment object" to capture. I succeeded, and with Guile 2.0, local-eval, procedure->syntax and the-environment were no more.

But no. As Guile releases started to make it into distributions, and users started to update their code, there arose such a howling on the mailing lists as set my hair on end. The ghost of local-eval was calling: it would not be laid to rest.

I resisted fate, for as long as I could do so in good conscience. In the end, Guile hacker Mark Weaver led an expedition to the mailing list moor, and came back with a plan.

Mark's plan was to have the syntax expander recognize the-environment, and residualize a form that would capture the identities of all lexical bindings. Like this:

(let ((x 10)) (the-environment))
=>
(let ((x 10))
  (make-lexical-environment
   ;; Procedure to wrap captured environment around
   ;; an expression
   wrapper
   ;; Captured variables: only "x" in this case
   (list (capture x))))

I'm taking it a little slow because hey, this is some tricky macrology. Let's look at (capture x) first. How do you capture a variable? In Scheme, with a closure. Like this:

;; Capture a variable with a closure.
;;
(define-syntax-rule (capture var)
  (case-lambda
    ;; When called with no arguments, return the value
    ;; of VAR.
    (() var)
    ;; When called with one argument, set the VAR to the
    ;; new value.
    ((new-val) (set! var new-val))))

The trickier part is reinstating the environment, so that x in a local-eval'd expression results in the invocation of a closure. Identifier syntax to the rescue:

;; The wrapper from above: a procedure that wraps
;; an expression in a lexical environment containing x.
;;
(lambda (exp)
  #`(lambda (x*) ; x* is a fresh temporary var
      (let-syntax ((x (identifier-syntax
                        (_ (x*))
                        ((set! _ val) (x* val)))))
        #,exp)))

By now it's clear what local-eval does: it wraps an expression, using the wrapper procedure from the environment object, evaluates that expression, then calls the resulting procedure with the case-lambda closures that captured the lexical variable.

So it's a bit intricate and nasty in some places, but hey, it finally tames the ghostly hound with modern Scheme. We were able to build local-eval on top of Guile's procedural macros, once a couple of accessors were added to our expander to return the set of bound identifiers visible in an expression, and to query whether those bindings were regular lexicals, or macros, or pattern variables, or whatever.

"watson, your service revolver, please."

As that Guile discussion was winding down, I started to hear the howls from an unexpected quarter: JavaScript. You might have heard, perhaps, that JavaScript eval is crazy. Well, it is. But ES5 strict was meant to kill off its most egregious aspect, in which eval can introduce new local variables to a function.

Now I've been slowly hacking on implementing block-scoped let and const in JavaScriptCore, so that we can consider switching gnome-shell over to use JSC. Beyond standard ES5 supported in JSC, existing gnome-shell code uses let, const, destructuring binding, and modules, all of which are bound to be standardized in the upcoming ES6. So, off to the hack.

My initial approach was to produce a correct implementation, and then make it fast. But the JSC maintainers, inspired by the idea that "let is the new var", wanted to ensure that let was fast from the beginning, so that it doesn't get a bad name with developers. OK, fair enough!

Beyond that, though, it looks like TC39 folk are eager to get let and const into all parts of JavaScript, not just strict mode. Do you hear the hound? It rides again! Now we have to figure out how block scope interacts with non-strict eval. Awooooo!

Thankfully, there seems to be a developing consensus that eval("let x = 20") will not introduce a new block-scoped lexical. So, down boy. The hound is at bay, for now.

life with dogs

I'm making my peace with eval. Certainly in JavaScript it's quite a burden for an implementor, but the current ES6 drafts don't look like they're making the problem worse. And in Scheme, I'm very happy to provide the primitives needed so that local-eval can be implemented in terms of our existing machinery, without needing symbol tables at runtime. But if you are making a new language, as you value your life, don't go walking on the local-eval moors at night!

Programming Praxis: String Rotation

2012-01-31T09:00:41+00:00

We have today another question from our never-ending supply of interview questions:

Write a function that takes two input strings and determines if one is a rotation of the other. For instance, “ProgrammingPraxis” and “PraxisProgramming” are rotations of each other, but “ProgrammingPrasix” is not a rotation of “ProgrammingPraxis” because the last three letters are out of order.

Your task is to write the requested function. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

Jeremy Kun: Handshake Lemma

2012-01-30T04:24:31+00:00

Problem: Prove or disprove: at a party of people, there must be an even number of people who have an odd number of friends at the party. Solution: Let be the set of all people, and for any person , … Continue reading →

Programming Praxis: Anagram Phrases

2012-01-27T09:00:43+00:00

Words that are formed from the same set of letters are anagrams of each other. For instance, pots, post, stop, spot, opts, and tops are anagrams. We studied anagrams in a previous exercise.

Anagrams can be extended from single words to phrases. For instance, “gin grammar prop six” and “maxim prong rasp rig” are anagrams for “programming praxis.”

Your task is to write a program to find all the anagram phrases for an input phrase that are present in a given dictionary; show only one permutation of each set of unique words. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

Alaric Snell-Pym: Alaric’s projects for this year

2012-01-26T17:09:46+00:00

This year's going to be pretty busy with settling into the new home, but I have a few projects. Finish the ring casting I nearly finished before the move. That's a priority. Resurrect my aluminium foundry. In particular, it's our bronze wedding anniversary, so Sarah's going to design a pattern for a sundial, which I will cast [...]

Programming Praxis: A Dozen Lines Of Code

2012-01-24T09:00:56+00:00

Today’s task will require your imagination and creativity.

A high-school programming teacher recently asked for examples of short programs with a high “cool” factor, the idea being to get his students interested in programming computers. I’m not sure the suggestions would work; today’s high-school students have been surrounded by computers their entire lives, and it takes a lot to make them think a program is cool. Being from a different generation, I can remember when I thought it was cool that a program properly skipped over the perforation on a stack of green-bar paper — many programs didn’t!

Your task is to write a cool program in a dozen lines of code. You can define cool in any way that you wish. Try not to abuse the definition of “line of code,” at least not too badly; to be concrete, we will say that your solution must not exceed 12 lines, and each line must not exceed 80 characters including white space. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

Grant Rettke: NexJ Scheme

2012-01-23T03:08:47+00:00

NexJ Scheme is an open source project providing an efficient and powerful interpreter for the programming language Scheme that executes in a Java virtual machine.

Today I was sort of shocked to learn that there is another implementation besides SISC and Kawa that runs on Java!

There wasn’t even an announcement for NexJ on comp.lang.scheme and NexJ has been around for two years :(! Rather it was mentioned on scheme-reports this week.

Jeremy Kun: The Fundamental Theorem of Algebra (with the Fundamental Group)

2012-01-22T20:46:13+00:00

This post assumes familiarity with some basic concepts in algebraic topology, specifically what a group is and the definition of the fundamental group of a topological space. The fundamental theorem of algebra has quite a few number of proofs (enough … Continue reading →

Programming Praxis: Knights On A Keypad

2012-01-20T09:00:18+00:00

Today’s exercise is an interview question that appeared on Stack Overflow a few years ago:

The numbers on a telephone keypad are arranged thus:

1 2 3 4 5 6 7 8 9 0

Starting from the digit 1, and choosing successive digits as a knight moves in chess, determine how many different paths can be formed of length n. There is no need to make a list of the paths, only to count them.

A knight moves two steps either horizontally or vertically followed by one step in the perpendicular direction; thus, from the digit 1 on the keypad a knight can move to digits 6 or 8, and from the digit 4 on the keypad a knight can move to digits 3, 9 or 0. A path may visit the same digit more than once.

Your task is to write a function that determines the number of paths of length n that a knight can trace on a keyboard starting from digit 1. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

Joe Marshall: A bit more challenging

2012-01-19T17:11:27+00:00

In my previous post, I gave the puzzle of taking a pattern and generating code that matches it. The tricky part was making sure that the pattern is completely traversed at compile time. If the object to be tested matches the pattern, the pattern variables and their values were to be returned in an alist.

It would be better, however, to generate code where the pattern variables become bindings of Scheme variables. Instead of generating code that stuffs the values into an alist, like this code does at the highlighted points:

(make-matcher '((? one) and (? two))) =>

(lambda (object)
  (and (pair? object)
       (let ((left-submatch
              ((lambda (object) (list (cons 'one object))) (car object)))
             (right-submatch
              ((lambda (object)
                 (and (pair? object)
                      (let ((left-submatch
                             ((lambda (object)
                                (and (eqv? object 'and)
                                     '()))
                              (car object)))
                            (right-submatch
                             ((lambda (object)
                                (and (pair? object)
                                     (let ((left-submatch
                                            ((lambda (object)
                                               (list (cons 'two object)))
                                             (car object)))
                                           (right-submatch
                                            ((lambda (object)
                                               (and (eqv? object '())
                                                    '()))
                                             (cdr object))))
                                       (and left-submatch
                                            right-submatch
                                            (append left-submatch
                                                    right-submatch)))))
                              (cdr object))))
                        (and left-submatch
                             right-submatch
                             (append left-submatch right-submatch)))))
               (cdr object))))
         (and left-submatch
              right-submatch
              (append left-submatch right-submatch)))))

We'd generate something more like this:

(make-matcher '((? one) and (? two)) <user code goes here>) =>

(lambda (object)
  (and (pair? object)
       (let ((one (car object))
             (tail1 (cdr object)))
         (and (pair? tail1)
              (eq? (car tail1) 'and)
              (let ((tail2 (cdr tail1)))
                (and (pair? tail2)
                     (let ((two (car tail2)))
                       (and (null? (cdr tail2))
                            <user code goes here>))))))))

This is more challenging for two reasons. First, we need to ensure that the pattern variable names become bound in a scope that encloses the user's code so that free references to pattern variables are correctly captured. In addition, we need to ensure that other “helper” bindings, like tail1 and tail2 do not capture free references by accident. (That is to say, watch your macro hygiene.) Second, you have to be sure that subpattern bindings are visible to the entire body of the user code. This will throw a monkey wrench into the simple recursive solution.