Planet Scheme

November 02, 2018

Greg Hendershott

Thread names

Sometimes people want Racket threads to have useful names — for example to show in logger output. Here is one way to do it.

When you call thread it returns a thread descriptor.

You may also get the thread descriptor of the current thread using current-thread.

The printed representation of a thread descriptor includes the object-name of its thunk procedure.

Often the thunk is an anonymous function. In that case the function object-name is something like /path/to/file.rkt:1:3 — so the thread descriptor prints as #<thread:/path/to/file.rkt:1:3>.

1
(thread (λ () #f)) ;=> #<thread:/tmp/thread.rkt:9:8>

It is more interesting when you name the thunk:

1
2
(define (foo) #f)
(thread foo) ;=> #<thread:foo>

You can also use object-name on the thread descriptor to extract the thunk’s object-name as a symbol:

1
2
(object-name (thread (λ () #f))) ;=> '/tmp/thread.rkt:18:21
(object-name (thread foo))       ;=> 'foo

What if youñ start 10 threads that all use the same thunk procedure? You can use procedure-rename to rename the thunk each time:

1
2
(thread (procedure-rename foo 'bar)) ;=> #<thread:bar>
(thread (procedure-rename foo 'baz)) ;=> #<thread:baz>

The new name can be any symbol — including one generated at runtime:

1
2
3
4
(define (now-sym) 
  (string->symbol (~a (current-inexact-milliseconds))))
(thread (procedure-rename foo (now-sym))) ;=> ex: #<thread:1541116698791.029>
(thread (procedure-rename foo (now-sym))) ;=> ex: #<thread:1541116698791.18>

Of course, you can include in the generated name any interesting identifying information from your problem domain — such as an “ID” or other details about a “job”, “request”, or “message”.

So something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
#lang at-exp racket/base

(require racket/format)

(define-logger thor)

(define (thor)
  (log-thor-debug @~a{hammering job @(object-name (current-thread))}))

(define (job-id) 
  (string->symbol (~a (current-inexact-milliseconds))))

(thread (procedure-rename thor (job-id)))
(thread (procedure-rename thor (job-id)))

Can produce logger output like this:

1
2
[  debug] thor: hammering job 1541119025888.965
[  debug] thor: hammering job 1541119025889.021

In conclusion, it is possible for threads to have unique names in Racket — provided you’re OK giving unique names to the thunks run by the theads.

by Greg Hendershott at November 02, 2018 12:10 AM

October 25, 2018

Programming Praxis

Car Crash

On Saturday morning I had a head-on collision at 75MPH, Medevac helicopter, serious internal bleeding, no broken bones. I spent four days in hospital, including one-and-a-half days in intensive care. I am home now, moving slowly but mostly okay.

I was alone in the car on an interstate highway early in the morning, heading to a weekend with friends away from home. About a minute ahead of me, an 18-wheel tractor/trailer rig hit a deer, lost control, jack-knifed and overturned, leaving the right lane blocked with the bottom of the tractor facing oncoming traffic. Because it was still astronomical twilight, with no moon and no street lamps, I did not see the tractor until it was within range of my headlights, and I was unable to avoid the collision, although I did partially swerve and struck only the right half of my car, from centerline to right side, spinning twice.

In the car about fifteen seconds behind me were four volunteer firefighters on their way to start their shift, so they were on the scene in seconds. They helped me out of the car, and I walked under my own power to the ambulance. The trainee driver of the truck was uninjured. The lead driver of the truck was asleep in the cab’s bed, and suffered a broken left leg.

My left ankle has a deep bruise, but is not broken, from striking the brake pedal; it is painful. My belly has a deep bruise from striking the steering wheel, and was the source of the internal bleeding; I take blood-thinning medication due to two prior pulmonary embolisms, so the bleeding was serious. I was flown from a local hospital to a regional first-class trauma center shortly after the crash. They monitored the bleeding, which stopped on its own after about two days without need for surgery (which was a real possibility). The bruise on my right belly is about twice the size of my right hand with fingers splayed wide, and is mostly black, with a few tinges of deep purple; it is not painful unless I bend or twist badly, which I have learned not to do. I am wearing a corset (the hospital calls it an abdominal binder) and taking a very low dose of pain-killers.

I’ve spoken to my boss; she even came to the hospital to visit and bring flowers. I’ll be away from work for at least several weeks, and away from blogging for just as long. I would appreciate it if my readers keep doing exercises while I am away; just look for any old exercises that strike your fancy.

by programmingpraxis at October 25, 2018 05:23 PM

October 19, 2018

Programming Praxis

Square Digit Chains

Today’s task appears on Project Euler, Rosetta Code, and Sloane’s, so it is well worth a look; we even did a version of this task in a previous exercise. Here is the Project Euler version of the problem:

A number chain is created by continuously added the square of the digits of a number to form a new number until it has been seen before. For example:

44 → 32 → 13 → 10 → 1 → 1

85 → 89 → 145 → 42 → 20 → 4 → 16 → 37 → 58 → 89

Therefore any chain that arrives at 1 or 89 will become stuck in an endless loop. What is most amazing is that EVERY starting number will eventually arrive at 1 or 89.

How many starting numbers below ten million will arrive at 89?

Your task is to write a program to find the count of starting numbers below ten million that arrive at 89, following the usual Project Euler rule that your computation may take no more than a minute of computation on a recent-vintage personal computer. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

by programmingpraxis at October 19, 2018 09:00 AM

October 16, 2018

Programming Praxis

Five Weekends

This task comes to us from Rosetta Code: The month of October 2010 has five Fridays, five Saturdays and five Sundays.

Your task is to count and make a list of all months from 1900 to 2100, inclusive, that have five Fridays, Saturdays and Sundays, and to count and make a list of all years from 1900 to 2100, inclusive, that have no such months. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

by programmingpraxis at October 16, 2018 09:00 AM

October 12, 2018

Programming Praxis

Boxing The Compass

Modern navigators use the 360° angles of a circle to point to compass directions, but traditional mariners use compass points with names like SW and ENE.

Your task is to write a program that converts from degrees to compass points. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

by programmingpraxis at October 12, 2018 09:00 AM

October 10, 2018

GNU Guix

A packaging tutorial for Guix

Introduction

GNU Guix stands out as the hackable package manager, mostly because it uses GNU Guile, a powerful high-level programming language, one of the Scheme dialects from the Lisp family.

Package definitions are also written in Scheme, which empowers Guix in some very unique ways, unlike most other package managers that use shell scripts or simple languages.

  • Use functions, structures, macros and all of Scheme expressiveness for your package definitions.

  • Inheritance makes it easy to customize a package by inheriting from it and modifying only what is needed.

  • Batch processing: the whole package collection can be parsed, filtered and processed. Building a headless server with all graphical interfaces stripped out? It's possible. Want to rebuild everything from source using specific compiler optimization flags? Pass the #:make-flags "..." argument to the list of packages. It wouldn't be a stretch to think Gentoo USE flags here, but this goes even further: the changes don't have to be thought out beforehand by the packager, they can be programmed by the user!

The following tutorial covers all the basics around package creation with Guix. It does not assume much knowledge of the Guix system nor of the Lisp language. The reader is only expected to be familiar with the command line and to have some basic programming knowledge.

A "Hello World" package

The “Defining Packages” section of the manual introduces the basics of Guix packaging. In the following section, we will partly go over those basics again.

GNU hello is a dummy project that serves as an idiomatic example for packaging. It uses the GNU build system (./configure && make && make install). Guix already provides a package definition which is a perfect example to start with. You can look up its declaration with guix edit hello from the command line. Let's see how it looks:

(define-public hello
  (package
    (name "hello")
    (version "2.10")
    (source (origin
              (method url-fetch)
              (uri (string-append "mirror://gnu/hello/hello-" version
                                  ".tar.gz"))
              (sha256
               (base32
                "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
    (build-system gnu-build-system)
    (synopsis "Hello, GNU world: An example GNU package")
    (description
     "GNU Hello prints the message \"Hello, world!\" and then exits.  It
serves as an example of standard GNU coding practices.  As such, it supports
command-line arguments, multiple languages, and so on.")
    (home-page "https://www.gnu.org/software/hello/")
    (license gpl3+)))

As you can see, most of it is rather straightforward. But let's review the fields together:

  • name: The project name. Using Scheme conventions, we prefer to keep it lower case, without underscore and using dash-separated words.
  • source: This field contains a description of the source code origin. The origin record contains these fields:

    1. The method, here url-fetch to download via HTTP/FTP, but other methods exist, such as git-fetch for Git repositories.
    2. The URI, which is typically some https:// location for url-fetch. Here the special mirror://gnu refers to a set of well known locations, all of which can be used by Guix to fetch the source, should some of them fail.
    3. The sha256 checksum of the requested file. This is essential to ensure the source is not corrupted. Note that Guix works with base32 strings, hence the call to the base32 function.
  • build-system: This is where the power of abstraction provided by the Scheme language really shines: in this case, the gnu-build-system abstracts away the famous ./configure && make && make install shell invocations. Other build systems include the trivial-build-system which does not do anything and requires from the packager to program all the build steps, the python-build-system, the emacs-build-system, and many more.
  • synopsis: It should be a concise summary of what the package does. For many packages a tagline from the project's home page can be used as the synopsis.
  • description: Same as for the synopsis, it's fine to re-use the project description from the homepage. Note that Guix uses Texinfo syntax.
  • home-page: Use HTTPS if available.
  • license: See guix/licenses.scm in the project source for a full list.

Time to build our first package! Nothing fancy here for now: we will stick to a dummy "my-hello", a copy of the above declaration.

As with the ritualistic "Hello World" taught with most programming languages, this will possibly be the most "manual" approach. We will work out an ideal setup later; for now we will go the simplest route.

Save the following to a file my-hello.scm.

(use-modules (guix packages)
             (guix download)
             (guix build-system gnu)
             (guix licenses))

(package
  (name "my-hello")
  (version "2.10")
  (source (origin
            (method url-fetch)
            (uri (string-append "mirror://gnu/hello/hello-" version
                                ".tar.gz"))
            (sha256
             (base32
              "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
  (build-system gnu-build-system)
  (synopsis "Hello, Guix world: An example custom Guix package")
  (description
   "GNU Hello prints the message \"Hello, world!\" and then exits.  It
serves as an example of standard GNU coding practices.  As such, it supports
command-line arguments, multiple languages, and so on.")
  (home-page "https://www.gnu.org/software/hello/")
  (license gpl3+))

We will explain the extra code in a moment.

Feel free to play with the different values of the various fields. If you change the source, you'll need to update the checksum. Indeed, Guix refuses to build anything if the given checksum does not match the computed checksum of the source code. To obtain the correct checksum of the package declaration, we need to download the source, compute the sha256 checksum and convert it to base32.

Thankfully, Guix can automate this task for us; all we need is to provide the URI:

$ guix download mirror://gnu/hello/hello-2.10.tar.gz

Starting download of /tmp/guix-file.JLYgL7
From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz...
following redirection to `https://mirror.ibcp.fr/pub/gnu/hello/hello-2.10.tar.gz'...
 …10.tar.gz  709KiB                                 2.5MiB/s 00:00 [##################] 100.0%
/gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i

In this specific case the output tells us which mirror was chosen. If the result of the above command is not the same as in the above snippet, update your my-hello declaration accordingly.

Note that GNU package tarballs come with an OpenPGP signature, so you should definitely check the signature of this tarball with gpg to authenticate it before going further:

$ guix download mirror://gnu/hello/hello-2.10.tar.gz.sig

Starting download of /tmp/guix-file.03tFfb
From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz.sig...
following redirection to `https://ftp.igh.cnrs.fr/pub/gnu/hello/hello-2.10.tar.gz.sig'...
 ….tar.gz.sig  819B                                                                                                                       1.2MiB/s 00:00 [##################] 100.0%
/gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig
0q0v86n3y38z17rl146gdakw9xc4mcscpk8dscs412j22glrv9jf
$ gpg --verify /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
gpg: Signature made Sun 16 Nov 2014 01:08:37 PM CET
gpg:                using RSA key A9553245FDE9B739
gpg: Good signature from "Sami Kerola <kerolasa@iki.fi>" [unknown]
gpg:                 aka "Sami Kerola (http://www.iki.fi/kerolasa/) <kerolasa@iki.fi>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 8ED3 96E3 7E38 D471 A005  30D3 A955 3245 FDE9 B739

Now you can happily run

$ guix package --install-from-file=my-hello.scm

You should now have my-hello in your profile!

$ guix package --list-installed=my-hello
my-hello    2.10    out
/gnu/store/f1db2mfm8syb8qvc357c53slbvf1g9m9-my-hello-2.10

We've gone as far as we could without any knowledge of Scheme. Now is the right time to introduce the minimum we need from the language before we can proceed.

A Scheme crash-course

As we've seen above, basic packages don't require much Scheme knowledge, if at all. But as you progress and your desire to write more and more complex packages grows, it will become both necessary and empowering to hone your Lisper skills.

Since an extensive Lisp course is very much out of the scope of this tutorial, we will only cover some basics here.

Guix uses the Guile implementation of Scheme. To start playing with the language, install it with guix package --install guile and start a REPL by running guile from the command line.

Alternatively you can also run guix environment --ad-hoc guile -- guile if you'd rather not have Guile installed in your user profile.

In the following examples we use the > symbol to denote the REPL prompt, that is, the line reserved for user input. See the Guile manual for more details on the REPL.

  • Scheme syntax boils down to a tree of expressions (or s-expression in Lisp lingo). An expression can be a literal such numbers and strings, or a compound which is a parenthesized list of compounds and literals. #t and #f stand for the booleans "true" and "false", respectively.

    Examples of valid expressions:

    > "Hello World!"
    "Hello World!"
    > 17
    17
    > (display (string-append "Hello " "Guix" "\n"))
    "Hello Guix!"
  • This last example is a function call embedded in another function call. When a parenthesized expression is evaluated, the first term is the function and the rest are the arguments passed to the function. Every function returns the last evaluated expression as value.

  • Anonymous functions are declared with the lambda term:

    > (lambda (x) (* x x))
    #<procedure 120e348 at <unknown port>:24:0 (x)>

    The above lambda returns the square of its argument. Since everything is an expression, the lambda expression returns an anonymous function, which can in turn be applied to an argument:

    > ((lambda (x) (* x x)) 3)
    9
  • Anything can be assigned a global name with define:

    > (define a 3)
    > (define square (lambda (x) (* x x)))
    > (square a)
    9
  • Procedures can be defined more concisely with the following syntax:

    (define (square x) (* x x))
  • A list structure can be created with the list procedure:

    > (list 2 a 5 7)
    (2 3 5 7)
  • The quote disables evaluation of a parenthesized expression: the first term is not called over the other terms. Thus it effectively returns a list of terms.

    > '(display (string-append "Hello " "Guix" "\n"))
    (display (string-append "Hello " "Guix" "\n"))
    > '(2 a 5 7)
    (2 a 5 7)
  • The quasiquote disables evaluation of a parenthesized expression until a comma re-enables it. Thus it provides us with fine-grained control over what is evaluated and what is not.

    > `(2 a 5 7 (2 ,a 5 ,(+ a 4)))
    (2 a 5 7 (2 3 5 7))

    Note that the above result is a list of mixed elements: numbers, symbols (here a) and the last element is a list itself.

  • Multiple variables can be named locally with let:

    > (define x 10)
    > (let ((x 2)
            (y 3))
        (list x y))
    (2 3)
    > x
    10
    > y
    ERROR: In procedure module-lookup: Unbound variable: y

    Use let* to allow later variable declarations to refer to earlier definitions.

    > (let* ((x 2)
             (y (* x 3)))
        (list x y))
    (2 6)
  • The keyword syntax is #:, it is used to create unique identifiers. See also the Keywords section in the Guile manual.

  • The percentage % is typically used for read-only global variables in the build stage. Note that it is merely a convention, like _ in C. Scheme Lisp treats % exactly the same as any other letter.

  • Modules are created with define-module. For instance

    (define-module (guix build-system ruby)
      #:use-module (guix store)
      #:export (ruby-build
                ruby-build-system))

    defines the module ruby which must be located in guix/build-system/ruby.scm somewhere in GUILE_LOAD_PATH. It depends on the (guix store) module and it exports two symbols, ruby-build and ruby-build-system.

For a more detailed introduction, check out Scheme at a Glance, by Steve Litt.

One of the reference Scheme books is the seminal Structure and Interpretation of Computer Programs, by Harold Abelson and Gerald Jay Sussman, with Julie Sussman. You'll find a free copy online, together with videos of the lectures by the authors. The book is available in Texinfo format as the sicp Guix package. Go ahead, run guix package --install sicp and start reading with info sicp (or with the Emacs Info reader). An unofficial ebook is also available.

You'll find more books, tutorials and other resources at https://schemers.org/.

Setup

Now that we know some Scheme basics we can detail the different possible setups for working on Guix packages.

There are several ways to set up a Guix packaging environment.

We recommend you work directly on the Guix source checkout since it makes it easier for everyone to contribute to the project.

But first, let's look at other possibilities.

Local file

This is what we previously did with my-hello. Now that we know more Scheme, let's explain the leading chunks. As stated in guix package --help:

-f, --install-from-file=FILE
                       install the package that the code within FILE
                       evaluates to

Thus the last expression must return a package, which is the case in our earlier example.

The use-modules expression tells which of the modules we need in the file. Modules are a collection of values and procedures. They are commonly called "libraries" or "packages" in other programming languages.

GUIX_PACKAGE_PATH

Note: Starting from Guix 0.16, the more flexible Guix "channels" are the preferred way and supersede GUIX_PACKAGE_PATH. See below.

It can be tedious to specify the file from the command line instead of simply calling guix package --install my-hello as you would do with the official packages.

Guix makes it possible to streamline the process by adding as many "package declaration paths" as you want.

Create a directory, say ~./guix-packages and add it to the GUIX_PACKAGE_PATH environment variable:

$ mkdir ~/guix-packages
$ export GUIX_PACKAGE_PATH=~/guix-packages

To add several directories, separate them with a colon (:).

Our previous my-hello needs some adjustments though:

(define-module (my-hello)
  #:use-module (guix licenses)
  #:use-module (guix packages)
  #:use-module (guix build-system gnu)
  #:use-module (guix download))

(define-public my-hello
  (package
    (name "my-hello")
    (version "2.10")
    (source (origin
              (method url-fetch)
              (uri (string-append "mirror://gnu/hello/hello-" version
                                  ".tar.gz"))
              (sha256
               (base32
                "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
    (build-system gnu-build-system)
    (synopsis "Hello, Guix world: An example custom Guix package")
    (description
     "GNU Hello prints the message \"Hello, world!\" and then exits.  It
serves as an example of standard GNU coding practices.  As such, it supports
command-line arguments, multiple languages, and so on.")
    (home-page "https://www.gnu.org/software/hello/")
    (license gpl3+)))

Note that we have assigned the package value to an exported variable name with define-public. This is effectively assigning the package to the my-hello variable so that it can be referenced, among other as dependency of other packages.

If you use guix package --install-from-file=my-hello.scm on the above file, it will fail because the last expression, define-public, does not return a package. If you want to use define-public in this use-case nonetheless, make sure the file ends with an evaluation of my-hello:

; ...
(define-public my-hello
  ; ...
  )

my-hello

This last example is not very typical.

Now my-hello should be part of the package collection like all other official packages. You can verify this with:

$ guix package --show=my-hello

Guix channels

Guix 0.16 features channels, which is very similar to GUIX_PACKAGE_PATH but provides better integration and provenance tracking. Channels are not necessarily local, they can be maintained as a public Git repository for instance. Of course, several channels can be used at the same time.

See the “Channels” section in the manual for setup details.

Direct checkout hacking

Working directly on the Guix project is recommended: it reduces the friction when the time comes to submit your changes upstream to let the community benefit from your hard work!

Unlike most software distributions, the Guix repository holds in one place both the tooling (including the package manager) and the package definitions. This choice was made so that it would give developers the flexibility to modify the API without breakage by updating all packages at the same time. This reduces development inertia.

Check out the official Git repository:

$ git clone https://git.savannah.gnu.org/git/guix.git

In the rest of this article, we use $GUIX_CHECKOUT to refer to the location of the checkout.

Follow the instruction from the "Contributing" chapter in the manual to set up the repository environment.

Once ready, you should be able to use the package definitions from the repository environment.

Feel free to edit package definitions found in $GUIX_CHECKOUT/gnu/packages.

The $GUIX_CHECKOUT/pre-inst-env script lets you use guix over the package collection of the repository.

  • Search packages, such as Ruby:

    $ cd $GUIX_CHECKOUT
    $ ./pre-inst-env guix package --list-available=ruby
        ruby    1.8.7-p374      out     gnu/packages/ruby.scm:119:2
        ruby    2.1.6   out     gnu/packages/ruby.scm:91:2
        ruby    2.2.2   out     gnu/packages/ruby.scm:39:2
  • Build a package, here Ruby version 2.1:

    $ ./pre-inst-env guix build --keep-failed ruby@2.1
    /gnu/store/c13v73jxmj2nir2xjqaz5259zywsa9zi-ruby-2.1.6
  • Install it to your user profile:

    $ ./pre-inst-env guix package --install ruby@2.1
  • Check for common mistakes:

    $ ./pre-inst-env guix lint ruby@2.1

Guix strives at maintaining a high packaging standard; when contributing to the Guix project, remember to

Once you are happy with the result, you are welcome to send your contribution to make it part of Guix. This process is also detailed in the manual.

It's a community effort so the more join in, the better Guix becomes!

Extended example

The above "Hello World" example is as simple as it goes. Packages can be more complex than that and Guix can handle more advanced scenarios. Let's look at another, more sophisticated package (slightly modified from the source):

(define-module (gnu packages version-control)
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (guix utils)
  #:use-module (guix packages)
  #:use-module (guix git-download)
  #:use-module (guix build-system cmake)
  #:use-module (gnu packages ssh)
  #:use-module (gnu packages web)
  #:use-module (gnu packages pkg-config)
  #:use-module (gnu packages python)
  #:use-module (gnu packages compression)
  #:use-module (gnu packages tls))

(define-public my-libgit2
  (let ((commit "e98d0a37c93574d2c6107bf7f31140b548c6a7bf")
        (revision "1"))
    (package
      (name "my-libgit2")
      (version (git-version "0.26.6" revision commit))
      (source (origin
                (method git-fetch)
                (uri (git-reference
                      (url "https://github.com/libgit2/libgit2/")
                      (commit commit)))
                (file-name (git-file-name name version))
                (sha256
                 (base32
                  "17pjvprmdrx4h6bb1hhc98w9qi6ki7yl57f090n9kbhswxqfs7s3"))
                (patches (search-patches "libgit2-mtime-0.patch"))
                (modules '((guix build utils)))
                (snippet '(begin
                            ;; Remove bundled software.
                            (delete-file-recursively "deps")
                            #t))))
      (build-system cmake-build-system)
      (outputs '("out" "debug"))
      (arguments
       `(#:tests? #t                            ; Run the test suite (this is the default)
         #:configure-flags '("-DUSE_SHA1DC=ON") ; SHA-1 collision detection
         #:phases
         (modify-phases %standard-phases
           (add-after 'unpack 'fix-hardcoded-paths
             (lambda _
               (substitute* "tests/repo/init.c"
                 (("#!/bin/sh") (string-append "#!" (which "sh"))))
               (substitute* "tests/clar/fs.h"
                 (("/bin/cp") (which "cp"))
                 (("/bin/rm") (which "rm")))
               #t))
           ;; Run checks more verbosely.
           (replace 'check
             (lambda _ (invoke "./libgit2_clar" "-v" "-Q")))
           (add-after 'unpack 'make-files-writable-for-tests
               (lambda _ (for-each make-file-writable (find-files "." ".*")))))))
      (inputs
       `(("libssh2" ,libssh2)
         ("http-parser" ,http-parser)
         ("python" ,python-wrapper)))
      (native-inputs
       `(("pkg-config" ,pkg-config)))
      (propagated-inputs
       ;; These two libraries are in 'Requires.private' in libgit2.pc.
       `(("openssl" ,openssl)
         ("zlib" ,zlib)))
      (home-page "https://libgit2.github.com/")
      (synopsis "Library providing Git core methods")
      (description
       "Libgit2 is a portable, pure C implementation of the Git core methods
provided as a re-entrant linkable library with a solid API, allowing you to
write native speed custom Git applications in any language with bindings.")
      ;; GPLv2 with linking exception
      (license license:gpl2))))

(In those cases were you only want to tweak a few fields from a package definition, you should rely on inheritance instead of copy-pasting everything. See below.)

Let's discuss those fields in depth.

git-fetch method

Unlike the url-fetch method, git-fetch expects a git-reference which takes a Git repository and a commit. The commit can be any Git reference such as tags, so if the version is tagged, then it can be used directly. Sometimes the tag is prefixed with a v, in which case you'd use (commit (string-append "v" version)).

To ensure that the source code from the Git repository is stored in a unique directory with a readable name we use (file-name (git-file-name name version)).

Note that there is also a git-version procedure that can be used to derive the version when packaging programs for a specific commit.

Snippets

Snippets are quoted (i.e. non-evaluated) Scheme code that are a means of patching the source. They are a Guix-y alternative to the traditional .patch files. Because of the quote, the code in only evaluated when passed to the Guix daemon for building.

There can be as many snippet as needed.

Snippets might need additional Guile modules which can be imported from the modules field.

Inputs

First, a syntactic comment: See the quasi-quote / comma syntax?

(native-inputs
 `(("pkg-config" ,pkg-config)))

is equivalent to

(native-inputs
 (list (list "pkg-config" pkg-config)))

You'll mostly see the former because it's shorter.

There are 3 different input types. In short:

  • native-inputs: Required for building but not runtime – installing a package through a substitute won't install these inputs.
  • inputs: Installed in the store but not in the profile, as well as being present at build time.
  • propagated-inputs: Installed in the store and in the profile, as well as being present at build time.

See the package reference in the manual for more details.

The distinction between the various inputs is important: if a dependency can be handled as an input instead of a propagated input, it should be done so, or else it "pollutes" the user profile for no good reason.

For instance, a user installing a graphical program that depends on a command line tool might only be interested in the graphical part, so there is no need to force the command line tool into the user profile. The dependency is a concern to the package, not to the user. Inputs make it possible to handle dependencies without bugging the user by adding undesired executable files (or libraries) to their profile.

Same goes for native-inputs: once the program is installed, build-time dependencies can be safely garbage-collected. It also matters when a substitute is available, in which case only the inputs and propagated inputs will be fetched: the native inputs are not required to install a package from a substitute.

Outputs

Just like how a package can have multiple inputs, it can also produce multiple outputs.

Each output corresponds to a separate directory in the store.

The user can choose which output to install; this is useful to save space or to avoid polluting the user profile with unwanted executables or libraries.

Output separation is optional. When the outputs field is left out, the default and only output (the complete package) is referred to as "out".

Typical separate output names include debug and doc.

It's advised to separate outputs only when you've shown it's worth it: if the output size is significant (compare with guix size) or in case the package is modular.

Build system arguments

The arguments is a keyword-value list used to configure the build process.

The simplest argument #:tests? can be used to disable the test suite when building the package. This is mostly useful when the package does not feature any test suite. It's strongly recommended to keep the test suite on if there is one.

Another common argument is :make-flags, which specifies a list of flags to append when running make, as you would from the command line. For instance, the following flags

#:make-flags (list (string-append "prefix=" (assoc-ref %outputs "out"))
                   "CC=gcc")

translate into

$ make CC=gcc prefix=/gnu/store/...-<out>

This sets the C compiler to gcc and the prefix variable (the installation directory in Make parlance) to (assoc-ref %outputs "out"), which is a build-stage global variable pointing to the destination directory in the store (something like /gnu/store/...-my-libgit2-20180408).

Similarly, it's possible to set the "configure" flags.

#:configure-flags '("-DUSE_SHA1DC=ON")

The %build-inputs variable is also generated in scope. It's an association table that maps the input names to their store directories.

The phases keyword lists the sequential steps of the build system. Typically phases include unpack, configure, build, install and check. To know more about those phases, you need to work out the appropriate build system definition in $GUIX_CHECKOUT/guix/build/gnu-build-system.scm:

(define %standard-phases
  ;; Standard build phases, as a list of symbol/procedure pairs.
  (let-syntax ((phases (syntax-rules ()
                         ((_ p ...) `((p . ,p) ...)))))
    (phases set-SOURCE-DATE-EPOCH set-paths install-locale unpack
            bootstrap
            patch-usr-bin-file
            patch-source-shebangs configure patch-generated-file-shebangs
            build check install
            patch-shebangs strip
            validate-runpath
            validate-documentation-location
            delete-info-dir-file
            patch-dot-desktop-files
            install-license-files
            reset-gzip-timestamps
            compress-documentation)))

Or from the REPL:

> (add-to-load-path "/path/to/guix/checkout")
> ,module (guix build gnu-build-system)
> (map first %standard-phases)
(set-SOURCE-DATE-EPOCH set-paths install-locale unpack bootstrap
patch-usr-bin-file patch-source-shebangs configure
patch-generated-file-shebangs build check install patch-shebangs strip
validate-runpath validate-documentation-location delete-info-dir-file
patch-dot-desktop-files install-license-files reset-gzip-timestamps
compress-documentation)

If you want to know more about what happens during those phases, consult the associated procedures.

For instance, as of this writing the definition of unpack for the GNU build system is

(define* (unpack #:key source #:allow-other-keys)
  "Unpack SOURCE in the working directory, and change directory within the
source.  When SOURCE is a directory, copy it in a sub-directory of the current
working directory."
  (if (file-is-directory? source)
      (begin
        (mkdir "source")
        (chdir "source")

        ;; Preserve timestamps (set to the Epoch) on the copied tree so that
        ;; things work deterministically.
        (copy-recursively source "."
                          #:keep-mtime? #t))
      (begin
        (if (string-suffix? ".zip" source)
            (invoke "unzip" source)
            (invoke "tar" "xvf" source))
        (chdir (first-subdirectory "."))))
  #t)

Note the chdir call: it changes the working directory to where the source was unpacked. Thus every phase following the unpack will use the source as a working directory, which is why we can directly work on the source files. That is to say, unless a later phase changes the working directory to something else.

We modify the list of %standard-phases of the build system with the modify-phases macro as per the list of specified modifications, which may have the following forms:

  • (add-before PHASE NEW-PHASE PROCEDURE): Run PROCEDURE named NEW-PHASE before PHASE.
  • (add-after PHASE NEW-PHASE PROCEDURE): Same, but afterwards.
  • (replace PHASE PROCEDURE).
  • (delete PHASE).

The PROCEDURE supports the keyword arguments inputs and outputs. Each input (whether native, propagated or not) and output directory is referenced by their name in those variables. Thus (assoc-ref outputs "out") is the store directory of the main output of the package. A phase procedure may look like this:

(lambda* (#:key inputs outputs #:allow-other-keys)
  (let (((bash-directory (assoc-ref inputs "bash"))
         (output-directory (assoc-ref outputs "out"))
         (doc-directory (assoc-ref outputs "doc"))
  ; ...
  #t)

The procedure must return #t on success. It's brittle to rely on the return value of the last expression used to tweak the phase because there is no guarantee it would be a #t. Hence the trailing #t to ensure the right value is returned on success.

Code staging

The astute reader may have noticed the quasi-quote and comma syntax in the argument field. Indeed, the build code in the package declaration should not be evaluated on the client side, but only when passed to the Guix daemon. This mechanism of passing code around two running processes is called code staging.

"Utils" functions

When customizing phases, we often need to write code that mimics the equivalent system invocations (make, mkdir, cp, etc.) commonly used during regular "Unix-style" installations.

Some like chmod are native to Guile. See the Guile reference manual for a complete list.

Guix provides additional helper functions which prove especially handy in the context of package management.

Some of those functions can be found in $GUIX_CHECKOUT/guix/guix/build/utils.scm. Most of them mirror the behaviour of the traditional Unix system commands:

  • which: Like the which system command.
  • find-files: Akin to the find system command.
  • mkdir-p: Like mkdir -p, which creates all parents as needed.
  • install-file: Similar to install when installing a file to a (possibly non-existing) directory. Guile has copy-file which works like cp.
  • copy-recursively: Like cp -r.
  • delete-file-recursively: Like rm -rf.
  • invoke: Run an executable. This should be used instead of system*.
  • with-directory-excursion: Run the body in a different working directory, then restore the previous working directory.
  • substitute*: A "sed-like" function.

Module prefix

The license in our last example needs a prefix: this is because of how the license module was imported in the package, as #:use-module ((guix licenses) #:prefix license:). The Guile module import mechanism gives the user full control over namespacing: this is needed to avoid clashes between, say, the zlib variable from licenses.scm (a license value) and the zlib variable from compression.scm (a package value).

Other build systems

What we've seen so far covers the majority of packages using a build system other than the trivial-build-system. The latter does not automate anything and leaves you to build everything manually. This can be more demanding and we won't cover it here for now, but thankfully it is rarely necessary to fall back on this system.

For the other build systems, such as ASDF, Emacs, Perl, Ruby and many more, the process is very similar to the GNU build system except for a few specialized arguments.

Learn more about build systems in

Programmable and automated package definition

We can't repeat it enough: having a full-fledged programming language at hand empowers us in ways that reach far beyond traditional package management.

Let's illustrate this with some awesome features of Guix!

Recursive importers

You might find some build systems good enough that there is little to do at all to write a package, to the point that it becomes repetitive and tedious after a while. A raison d'être of computers is to replace human beings at those boring tasks. So let's tell Guix to do this for us and create the package definition of an R package from CRAN (the output is trimmed for conciseness):

$ guix import cran --recursive walrus

(define-public r-mc2d
    ; ...
    (license gpl2+)))

(define-public r-jmvcore
    ; ...
    (license gpl2+)))

(define-public r-wrs2
    ; ...
    (license gpl3)))

(define-public r-walrus
  (package
    (name "r-walrus")
    (version "1.0.3")
    (source
      (origin
        (method url-fetch)
        (uri (cran-uri "walrus" version))
        (sha256
          (base32
            "1nk2glcvy4hyksl5ipq2mz8jy4fss90hx6cq98m3w96kzjni6jjj"))))
    (build-system r-build-system)
    (propagated-inputs
      `(("r-ggplot2" ,r-ggplot2)
        ("r-jmvcore" ,r-jmvcore)
        ("r-r6" ,r-r6)
        ("r-wrs2" ,r-wrs2)))
    (home-page "https://github.com/jamovi/walrus")
    (synopsis "Robust Statistical Methods")
    (description
      "This package provides a toolbox of common robust statistical tests, including robust descriptives, robust t-tests, and robust ANOVA.  It is also available as a module for 'jamovi' (see <https://www.jamovi.org> for more information).  Walrus is based on the WRS2 package by Patrick Mair, which is in turn based on the scripts and work of Rand Wilcox.  These analyses are described in depth in the book 'Introduction to Robust Estimation & Hypothesis Testing'.")
    (license gpl3)))

The recursive importer won't import packages for which Guix already has package definitions, except for the very first.

Not all applications can be packaged this way, only those relying on a select number of supported systems. Read about the full list of importers in the guix import section of the manual.

Automatic update

Guix can be smart enough to check for updates on systems it knows. It can report outdated package definitions with

$ guix refresh hello

In most cases, updating a package to a newer version requires little more than changing the version number and the checksum. Guix can do that automatically as well:

$ guix refresh hello --update

Inheritance

If you've started browsing the existing package definitions, you might have noticed that a significant number of them have a inherit field:

(define-public adwaita-icon-theme
  (package (inherit gnome-icon-theme)
    (name "adwaita-icon-theme")
    (version "3.26.1")
    (source (origin
              (method url-fetch)
              (uri (string-append "mirror://gnome/sources/" name "/"
                                  (version-major+minor version) "/"
                                  name "-" version ".tar.xz"))
              (sha256
               (base32
                "17fpahgh5dyckgz7rwqvzgnhx53cx9kr2xw0szprc6bnqy977fi8"))))
    (native-inputs
     `(("gtk-encode-symbolic-svg" ,gtk+ "bin")))))

All unspecified fields are inherited from the parent package. This is very convenient to create alternative packages, for instance with different source, version or compilation options.

Getting help

Sadly, some applications can be tough to package. Sometimes they need a patch to work with the non-standard filesystem hierarchy enforced by the store. Sometimes the tests won't run properly. (They can be skipped but this is not recommended.) Other times the resulting package won't be reproducible.

Should you be stuck, unable to figure out how to fix any sort of packaging issue, don't hesitate to ask the community for help.

See the Guix homepage for information on the mailing lists, IRC, etc.

Conclusion

This tutorial was an showcase of the sophisticated package management that Guix boasts. At this point we have mostly restricted this introduction to the gnu-build-system which is a core abstraction layer on which more advanced abstractions are based.

Now where do we go from here? Next we ought to dissect the innards of the build system by removing all abstractions, using the trivial-build-system: this should give us a thorough understanding of the process before investigating some more advanced packaging techniques and edge cases.

Other features worth exploring are the interactive editing and debugging capabilities of Guix provided by the Guile REPL.

Those fancy features are completely optional and can wait; now is a good time to take a well-deserved break. With what we've introduced here you should be well armed to package lots of programs. You can get started right away and hopefully we will see your contributions soon!

References

Notices

CC-BY-SA

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

About GNU Guix

GNU Guix is a transactional package manager for the GNU system. The Guix System Distribution or GuixSD is an advanced distribution of the GNU system that relies on GNU Guix and respects the user's freedom.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. Guix uses low-level mechanisms from the Nix package manager, except that packages are defined as native Guile modules, using extensions to the Scheme language. GuixSD offers a declarative approach to operating system configuration management, and is highly customizable and hackable.

GuixSD can be used on an i686, x86_64, ARMv7, and AArch64 machines. It is also possible to use Guix on top of an already installed GNU/Linux system, including on mips64el and aarch64.

by Pierre Neidhardt at October 10, 2018 02:00 PM

October 09, 2018

Programming Praxis

Babbage’s Number

Charles Babbage, whose Analytical Engine was a direct predecessor of today’s digital computer, gave this example of a problem that his Analytical Engine could solve in an 1837 letter to Lord Bowden:

What is the smallest positive integer whose square ends in the digits 269,696?

Babbage knew that 99,736 has a square with the required ending, but didn’t know if there was a smaller number.

Your task is to find Babbages’s number. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

by programmingpraxis at October 09, 2018 09:00 AM

October 05, 2018

Programming Praxis

Leap Year

Today’s exercise comes from a student who asked for homework help on the internet:

Write a program that determines whether or not a given year is a leap year. A year is a leap year if it is divisible by 4, except that it is not a leap year if it is divisible by 100, except that it is a leap year if it is divisible by 400.

Your task is to write a program to identify leap years. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

by programmingpraxis at October 05, 2018 09:00 AM

October 04, 2018

GNU Guix

Join GNU Guix through Outreachy

We are happy to announce that for the second time this year, GNU Guix offers a three-month internship through Outreachy, the inclusion program for groups traditionally underrepresented in free software and tech. We currently propose one subject to work on:

  1. Create user video documentation for GNU Guix.

Eligible persons should apply by November 22th.

If you’d like to contribute to computing freedom, Scheme, functional programming, or operating system development, now is a good time to join us. Let’s get in touch on the mailing lists and on the #guix channel on the Freenode IRC network!

About GNU Guix

GNU Guix is a transactional package manager for the GNU system. The Guix System Distribution or GuixSD is an advanced distribution of the GNU system that relies on GNU Guix and respects the user's freedom.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. Guix uses low-level mechanisms from the Nix package manager, except that packages are defined as native Guile modules, using extensions to the Scheme language. GuixSD offers a declarative approach to operating system configuration management, and is highly customizable and hackable.

GuixSD can be used on i686, x86_64, and armv7 machines. It is also possible to use Guix on top of an already installed GNU/Linux system, including on mips64el and aarch64.

by Gábor Boskovits at October 04, 2018 08:00 AM

October 03, 2018

Greg Hendershott

racket-mode

Even though it’s been one of my most time-consuming projects, I’ve hardly blogged about racket-mode — an Emacs major mode for Racket. To change that, here’s a post giving an overview of how racket-mode works, as well as a look at how it might grow someday.

Cast of characters

racket-mode consists of two main parts:

  • An Emacs Lisp “front end”. This implements various major and minor modes, the main two of which are:

    • racket-mode for editing .rkt files

    • racket-repl-mode for a REPL

  • A Racket “back end”. This is a racket process that runs the back end code, which in turn runs your program and implements “commands”, as discussed below.

This mix of Emacs Lisp and Racket code is delivered as an Emacs package (e.g. via MELPA), where the Racket files come along for the ride.1

Crossing the chasm

The Emacs front end (“client”) sends command requests to the Racket back end (“server”) which sends command responses. Exactly how this happens is the aspect of racket-mode that has changed the most over the years.

Historical

The very earliest version of racket-mode was a very thin wrapper around XREPL, which lets you enter commands prefixed by a comma2 like ,enter <filename>. So racket-mode would mostly just comint-send-string a few of these commands (invisibly type them into the buffer on your behalf).

But.

  • What if the user’s program does read or read-line? If we send commands to the back end, the user’s program will get them.

  • Later, command responses became more important. For example, racket-mode’s racket-describe command returns HTML (from Racket’s documentation) to be shown in an Emacs buffer. Well, occasionally the HTML might contain something like "\ndiv>" — breaking code trying to read a response up to the next REPL prompt.

Long story short: If you like edge cases, you’ll love multiplexing textual command I/O with user program I/O. That approach is an easy way to start casually. But it’s actually easier long-term to move the command I/O “out-of-band”, when possible. It sucks to demux.

Today

Early Summer 2018 I again changed how command requests and responses work.

The Racket back end starts a TCP “command server”, and the Emacs front end connects to it.

Although the connection method is TCP, the protocol isn’t HTTP. Instead, the command server:

  • Accepts a single TCP connection at a time (multiple requests and responses are sent over the one connection)

  • Uses a subset of valid Emacs Lisp s-expressions for requests and responses.

Command requests are (nonce command param ...).

A thread is spun off to handle each request, so that a long-running command won’t block others. The nonce supplied with the request is returned with the response, so that the client can match the response with the request.3

Command responses are either (nonce 'ok sexp ...+) or (nonce 'error "message").

Here’s a code snippet to give a rough sense of the commands:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
  (match sexpr
    [`(run ,what ,mem ,pp? ,ctx ,args ,dbg) (run what mem pp? ctx args dbg)]
    [`(path+md5)                            (cons (or path 'top) md5)]
    [`(syms)                                (syms)]
    [`(def ,str)                            (find-definition str)]
    [`(mod ,sym)                            (find-module sym maybe-mod)]
    [`(describe ,str)                       (describe str)]
    [`(doc ,str)                            (doc str)]
    [`(type ,v)                             (type v)]
    [`(macro-stepper ,str ,into-base?)      (macro-stepper str into-base?)]
    [`(macro-stepper/next)                  (macro-stepper/next)]
    [`(requires/tidy ,reqs)                 (requires/tidy reqs)]
    [`(requires/trim ,path-str ,reqs)       (requires/trim path-str reqs)]
    [`(requires/base ,path-str ,reqs)       (requires/base path-str reqs)]
    [`(find-collection ,str)                (find-collection str)]
    [`(get-profile)                         (get-profile)]
    [`(get-uncovered)                       (get-uncovered path)]
    [`(check-syntax ,path-str)              (check-syntax path-str)]
    [`(eval ,v)                             (eval-command v)]
    [`(repl-submit? ,str ,eos?)             (repl-submit? submit-pred str eos?)]
    [`(debug-eval ,src ,l ,c ,p ,code)      (debug-eval src l c p code)]
    [`(debug-resume ,v)                     (debug-resume v)]
    [`(debug-disable)                       (debug-disable)]
    [`(exit)                                (exit)])

This approach also makes it reasonably simple for the Emacs Lisp front end to issue some commands “asynchronously”. In fact the lowest-level command function is async:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
(defvar racket--cmd-nonce->callback (make-hash-table :test 'eq)
  "A hash from nonce to callback function.")
(defvar racket--cmd-nonce 0
  "Increments for each command request we send.")

(defun racket--cmd/async-raw (command-sexpr &optional callback)
  "Send COMMAND-SEXPR and return. Later call CALLBACK with the response sexp.

If CALLBACK is not supplied or nil, defaults to `ignore'."
  (racket--repl-ensure-buffer-and-process nil)
  (racket--cmd-connect-finish)
  (cl-incf racket--cmd-nonce)
  (when (and callback
             (not (equal callback #'ignore)))
    (puthash racket--cmd-nonce callback racket--cmd-nonce->callback))
  (process-send-string racket--cmd-proc
                       (format "%S\n" (cons racket--cmd-nonce
                                            command-sexpr))))

Code that doesn’t need to examine error responses can use a simplified wrapper:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
(defun racket--cmd/async (command-sexpr &optional callback)
  "You probably want to use this instead of `racket--cmd/async-raw'.

CALLBACK is only called for 'ok responses, with (ok v ...)
unwrapped to (v ...). 'error responses are handled here."
  (let ((buf (current-buffer)))
    (racket--cmd/async-raw
     command-sexpr
     (if callback
         (lambda (response)
           (pcase response
             (`(ok ,v)    (with-current-buffer buf (funcall callback v)))
             (`(error ,m) (message "%s" m))
             (v           (message "Unknown command response: %S" v))))
       #'ignore))))

Code that needs to wait synchronously for a response can use another wrapper:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
(defun racket--cmd/await (command-sexpr)
  "Send COMMAND-SEXPR. Await and return an 'ok response value, or raise `error'."
  (let* ((awaiting 'RACKET-REPL-AWAITING)
         (response awaiting))
    (racket--cmd/async-raw command-sexpr
                           (lambda (v) (setq response v)))
    (with-timeout (racket-command-timeout
                   (error "racket-command process timeout"))
      (while (eq response awaiting)
        (accept-process-output nil 0.001))
      (pcase response
        (`(ok ,v)    v)
        (`(error ,m) (error "%s" m))
        (v           (error "Unknown command response: %S" v))))))

In other words, asynchronous is the baseline. Synchronous is the special case where the callback setqs the response and the main Emacs thread polls for it.

What does it mean to “run” a file

The main command is run, which causes it to run a specific .rkt file. “Run” means something similar to what it does in DrRacket: dynamic-require a relative module path and use module->namespace so that you are “inside” the module and can see all its module-level definitions (even those it does not provide). Then it does a read-eval-print-loop, which is the REPL you see in the racket-repl-mode buffer.

Importantly, each run uses a fresh custodian and namespace. Therefore each run discards any changes you made only in the REPL and “resets” you to a namespace corresponding to the .rkt file. In other words, the single source of truth is your source file.

One handy nuance: If, when you run, point4 is inside a submodule at any depth, you enter that submodule. So for example you can put point within a (module+ test ___) form and C-c C-c. Not only will this run your tests, the REPL will be “inside” the test submodule so you can explore and experiment more. Or you can run a main, or any other, submodule.

Of course there are many wrinkles:

  • In some cases we “instrument” the module to support features like profiling, test coverage, or step debugging.

  • We look for and run configure-runtimes or get-infos supplied by a #lang.

  • We warn when a #lang doesn’t supply #%top-interaction.

  • We load racket/gui lazily. (That way, you don’t get a pointless GUI frame window while working on textual programs.)

  • We maybe set up custom print handlers to use pretty-print.

  • If the file contains syntax errors, we try to distill it to a “skeleton” of things that create runtime, module-level bindings, i.e. requires and defines. You can think of this as replacing all (define id rhs) with (define id (void)). Why? That way, we get a namespace with the definition identifiers available to things like auto-complete.

  • And on and on.

Summer two step

Over the summer of 2018, I added a couple features under the theme of “stepping”.

Macro step expander

A new racket-stepper-mode presents a consistent diff -U 3 format UI for both:

  • expand-once-ing expressions

  • whole-file expansion using macro-debugger/stepper-text (although the UI is not as ambitious as the fantastic macro stepper in DrRacket)

Whole-file expansion potentially can take many seconds. So this is an example where it’s good to have the ability for command responses to come as an asynchronous callback. Emacs isn’t “frozen” while waiting.

Step debugger

Also new is an interactive step debugger. I won’t repeat the commit message or the documentation here. I’ll just mention that, as the commit message says, I had started to work on this several years ago but decided not to merge it. The new async command protocol was one factor that made it easier. Also, by labeling it “experimental”, I was less worried should it turn out to be too much hassle to support long-term.

I’m curious to see how much people actually use it. I’ve found that I want an interactive debugger much less often writing mostly functional Racket than I did when writing imperative C/C++. With Racket, a step debugger feels more like a nice-to-have than a must-have. Even so, while working in Emacs Lisp for racket-mode, sometimes I have found edebug to be handy. So we’ll see.

racket-mode step debugger

Future

Modulo bugs and some UX refinements, I’d say that racket-mode has reached the point where it’s a pretty good "#lang-racket-mode". That is, it’s pretty good for working with s-expression #langs like racket, racket/base, typed/racket, and so on.

However it does not attempt to be a “racket-is-a-programming-language-programming-language-mode” like DrRacket. Maybe that is where it should head next. If so, I’m aware of three main areas, only one of which is done.

Submitted for your approval, in the REPL zone

A #lang may supply a drracket:submit-predicate to say whether some chunk of text is a complete expression. In other words, when a user hits ENTER in the REPL, should this be sent for evaluation, or, simply insert a newline and wait for them to supply more?

As of a couple months ago, racket-mode does actually look for and use this.

Syntax highlighting

“Font-lock” is what Emacs calls the process of changing the appearance of text — such as syntax highlighting for programming modes. Mostly this is done via regular expressions, although an arbitrary function can be supplied to do both fancier matching and altering.

Traditionally font-lock distinguishes between language elements that are “keywords” vs. “builtins”. When it comes to Racket, this is sort of a weird distinction that I handled ad hoc.

DrRacket takes a different approach. Each #lang may supply a color-lexer. Clearly this is the Correct Way.

And yet. One great way to make Emacs feel sluggish is to do font-lock poorly. To-date, I’ve been worried about the performance of trying to use color-lexers, which are color:text<%> interfaces. How will this perform marshaled over the TCP connection? Will this force racket-mode to use racket/gui always, instead of only when user programs need it?

Indentation

Similarly, racket-mode could someday use a #lang-supplied drracket:indentation — and similarly, naive indentation can bring Emacs to its knees.

Currently racket-mode indents using Emacs Lisp that is mostly hardcoded with special handling for known macros5 in some popular #langs.

I say “mostly hardcoded” because it is extensible in the tradition of lisp-mode and scheme-mode: racket-mode looks for a 'racket-indent-function property on an Emacs Lisp symbol of the same name as the Racket macro. racket-mode sets up some of these itself. You can add your own in your Emacs init file:

1
(put 'racket-macro-name 'racket-indent-function indent-value)

You can also do this using Emacs Directory Variables or in .rkt files using Emacs File Variables. An example of the latter setting up indentation for some sql package syntax:

1
2
3
4
5
6
;; Local Variables:
;; eval: (put 'insert 'racket-indent-function 'defun)
;; eval: (put 'update 'racket-indent-function 'defun)
;; eval: (put 'delete 'racket-indent-function 'defun)
;; eval: (put 'select 'racket-indent-function 'defun)
;; End:

Although this works, it’s not ideal.

So it would be nice for racket-mode to use a #lang-supplied drracket:indentation.

However, even that doesn’t seem enough: Not just a full-blown #lang, but really any library module — like sql — ought to be able to specify indent for special syntax.

So I don’t yet know a good story for rich indentation that is both complete and performant.

Conclusion

Hopefully this gives you some overview of how racket-mode works, as well as what it currently does or does not attempt to do, and why.

  1. Another approach would be to split it into an Emacs package and a Racket package. This would have various pros and cons. I think the main disadvantage is that users would need both to update to stay in sync. When using CIDER for Clojure I found this a little troublesome. 

  2. Since , is reader shorthand for unquote, which is only valid inside a quasiquote, this is a clever way to distinguish a command from a valid Racket expression. 

  3. The nonce needn’t be a cryptographically secure random number, just unique; an increasing integer is fine. 

  4. “Point” is Emacs’ name for what you might call the cursor or caret. 

  5. As far as I can tell, the convention is that function applications should be indented consistently — only macro invocations might use custom indent. 

by Greg Hendershott at October 03, 2018 04:00 AM

October 02, 2018

Programming Praxis

Palindrome List

Today’s exercise sounds like a homework problem:

Write a program that determines if a linked list of integers is palindromic — i.e., reads the same in both directions. Your solution must operate in O(1) space.

Your task is to write a program to determine if a list of integers is palindromic, in O(1) space. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

by programmingpraxis at October 02, 2018 09:00 AM

September 29, 2018

GNU Guix

Upcoming Talk: "Everyday Use of GNU Guix"

At SeaGL 2018, Chris Marusich will present a talk introducing GNU Guix to people of all skill levels and backgrounds. SeaGL is an annual GNU/Linux conference in Seattle. Attendance is gratis.

If you're in the Seattle area, please consider coming! Even if you can't make it in person, the talk will be recorded and later made available on the SeaGL website, so you can watch it at your convenience after it's been uploaded.

Abstract

Everyday Use of GNU Guix

In this talk, I will introduce GNU Guix: a liberating, dependable, and hackable package manager that follows the "purely functional software deployment model" pioneered by Nix.

I will demonstrate some common use cases of Guix and show you how I use it in my everyday life. In addition, I will briefly explain the basic idea behind the functional model and how it enables Guix to provide useful features like the following:

  • Transactional upgrades and roll-back of installed software.
  • Unprivileged users can simultaneously install multiple versions of software.
  • Transparently build from source or download pre-built binaries.
  • Installed software is bootstrappable, trustable, and auditable all the way down to your compiler's compiler.
  • Eliminates an entire class of "works on my system" type problems.

No prior knowledge of Guix, Nix, or the functional model is required. When you leave this talk, I hope you will have a basic understanding of what Guix is, how to use it, and why it will help make your life brighter.

Schedule

The talk will take place at the following time and location:

For details, please refer to the official SeaGL page for the talk.

About GNU Guix

GNU Guix is a transactional package manager for the GNU system. The Guix System Distribution or GuixSD is an advanced distribution of the GNU system that relies on GNU Guix and respects the user's freedom.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. Guix uses low-level mechanisms from the Nix package manager, except that packages are defined as native Guile modules, using extensions to the Scheme language. GuixSD offers a declarative approach to operating system configuration management, and is highly customizable and hackable.

GuixSD can be used on an i686, x86_64 and armv7 machines. It is also possible to use Guix on top of an already installed GNU/Linux system, including on mips64el and aarch64.

by Chris Marusich at September 29, 2018 03:00 PM

September 28, 2018

Programming Praxis

No Exercise Today

There was no exercise Monday, and there is no exercise today. I’ve been very busy at work on the implementation of a major upgrade to our primary application, with go-live scheduled for early December. I’m responsible for a major component of the system, which unfortunately hasn’t yet worked properly. I’ll be sure to write an exercise this weekend for next Tuesday.

by programmingpraxis at September 28, 2018 09:00 AM

September 21, 2018

Programming Praxis

The 37% Rule

You have to hire a new programmer from a pool of 100 applicants. One method is to interview all 100 and hire the best, but that takes a while. Mathematicians have developed the 37% rule:

Examine 37% of the candidates, knowing in advance you will not select any of them. Then examine the remaining candidates one-by-one, choosing the first of them that is better than any of the first 37%.

Your task is to simulate this process and determine how close you get to the optimal candidate. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

by programmingpraxis at September 21, 2018 09:00 AM

September 18, 2018

Programming Praxis

1-800-PPRAXIS

I hate businesses with telephone numbers that spell words; it’s awkward to type letters at a telephone, and often the words are abbreviated or spelled wrong so they are no longer mnemonic, which defeats the purpose. So this morning when I had to call one of those businesses, I wrote a program to translate letters to numbers after I had been waiting on hold long enough to get really annoyed.

Your task is to write a program to translate telephone numbers specified as words. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

by programmingpraxis at September 18, 2018 09:00 AM