From its inception, Guix has had a thorough test suite—something that’s not only reassuring, but also the thing that allows for fearless evolution of the code. That we didn’t have this safety net when hacking on the whole operating system, GuixSD, made it uncomfortable and more risky. We are now addressing the problem with the introduction of *system tests*, closing one of the major roadblocks towards 1.0.

Before going into details, let me recap the sorts of testing that already occurred in Guix land.

Guix’s test suite currently contains almost 600 *unit tests*. Each one of these stresses one particular function or subset of the functionality of Guix. This covers core package management functionality such as package builds, utility modules such as monads or the public key infrastructure (PKI) used for authenticating binaries, maintenance tools such as lint and the importers, as well as the command-line interface.

Since Guix provides Scheme modules for use both in the package management front-end and on the “build side”, the latter is also tested. This includes part of the build systems, and helpers like our ELF validation module.

Then come the software packages that Guix ships. All of the packages in the distro are under continuous integration on the 4 supported architectures (32-bit and 64-bit Intel compatible, as well as MIPS64 and ARMv7.) Our build farm serves the resulting binaries, which users can choose to download as substitutes for local builds. Our build server, which currently runs an instance of Hydra, always shows the number of succeeding/failing builds on its dashboard. That way, breakage introduced by changes in the package collection is always rapidly detected. This is a direct benefit of the functional packaging model.

Additionally, our policy is to always run each package’s test suite (typically “make check”) as part of its build process, unless there is a serious technical obstacle to doing that. That way, we can, and do catch integration issues, incompatibilities, and plain bugs before they hit users.

So far, so good. Now, what about GuixSD itself? GuixSD did not have an automated test suite until now. What it did have, though, is the ability to instantiate an operating system in a virtual machine (VM) or in a container. You would write your operating system declaration in a file, then run, say:

This gives you a script to launch a VM running an instance of the OS declared in ‘my-config.scm’. Already pretty convenient! And indeed, even more so back in the days when we were eating a fair amount of dog food. In fact, that’s how we ate our first dog food dishes, and the VM infrastructure was the fork and knife that made it more tolerable.

So what could we test exactly? Roughly, we want to test that the instantiated system behaves according to the source ‘operating-system’ declaration: that user accounts are all there, that system services are running as expected, that all of the configuration is taken into account.

To do that, we need to run the system under test in a VM, but we also need to instrument it. We use QEMU to run our VMs, and QEMU along with the Linux virtio-serial module nicely supports communication between the guest operating system and the host, a strategy also used by NixOS’ test driver. Concretely, we define a “marionette” service, which hooks a Guile read-eval-print loop (REPL) inside the guest. This allows the host to evaluate arbitrary code in the guest VM.

Now we can write build processes (aka. “derivations”) that will:

- instantiate an instrumented variant of the operating system configuration we want to test in a VM image;
- spawn the VM, run a series of tests on the guest OS, and return the test results.

Thus, a system test to make sure the ‘uname’ system call returns something that matches the OS declaration looks like this:

There are interesting things going on here. First, while this is all Scheme code, there are in fact three tiers or strata of code at play here: the code that produces the OS declaration and the derivation, the build code of that derivation—the test—embodied in a g-expression, and code sent from the host to the guest VM via ‘marionette-eval’.

Using Scheme all the way means we can share code, use the right tools such as the SRFI-64 test framework here, pass values from one tier to another, and so on. And of course, being a full-blown language rather than shell scripts or similar means we have a rich and semantically-clear interface at our fingertips: we can directly access the data structures that matter rather than grepping the output of high-level commands. As an example, we can directly query the system service manager right from Scheme, which is often useful in system tests.

Guix now includes the test infrastructure described above; running “make check-system” runs all the currently defined tests. Currently we have tests for basic functionality. This includes making sure that all the system services declared are available and running. We have tests for specific system services such as the mcron job scheduling daemon—inspired by your parents’ cron, but with a powerful Scheme interface—and Avahi and its name service switch (NSS) integration.

Last but not least, we have tests of the full GuixSD installation procedure, which turned out to be more involved than the other tests. This works by running the GuixSD installation image in a VM, using another VM image as the target installation media, and finally booting the newly-installed system.

All the tests are automatically run on our build farm (see here, here, or there), which provides quick feedback. One step closer to 1.0!

GNU Guix is a transactional package manager for the GNU system. The Guix System Distribution or GuixSD is an advanced distribution of the GNU system that relies on GNU Guix and respects the user's freedom.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. Guix uses low-level mechanisms from the Nix package manager, except that packages are defined as native Guile modules, using extensions to the Scheme language. GuixSD offers a declarative approach to operating system configuration management, and is highly customizable and hackable.

GuixSD can be used on an i686 or x86_64 machine. It is also possible to use Guix on top of an already installed GNU/Linux system, including on mips64el and armv7.

Today’s exercise is in the style of Project Euler, so the rules are that your solution must not use more than a minute of computer time and that you can’t peek at the solution until you have the answer yourself. Some numbers have the curious property that when they are squared, the number appears in […]

Today’s exercise is in the style of Project Euler, so the rules are that your solution must not use more than a minute of computer time and that you can’t peek at the solution until you have the answer yourself.

Some numbers have the curious property that when they are squared, the number appears in the least-significant digits of the product. For instance, 625

^{2}= 390625, and 7106376^{2}= 50543227109376.What is the sum of all numbers less than 10

^{20}that have this curious property?

Your task is to find the sum described above. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

John Cook is a mathematician and programmer who runs a fascinating blog that I frequent. Cook recently had an article about the prime factors of telephone numbers. He explained that, for 10-digit telephone numbers as used in the United States, the average number of distinct prime factors is 3.232 and the distribution is between 1 […]

John Cook is a mathematician and programmer who runs a fascinating blog that I frequent.

Cook recently had an article about the prime factors of telephone numbers. He explained that, for 10-digit telephone numbers as used in the United States, the average number of distinct prime factors is 3.232 and the distribution is between 1 and 5 distinct prime factors about 73% of the time.

Your task is to write a function that determines the number of distinct prime factors of a number, and use that function to explore the distribution of the number of distinct prime factors in a range of telephone numbers. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

I like to read a web site called Career Cup, both to enjoy solving some of the programming exercise given there and to find exercise for Programming Praxis. As I write this exercise, here are the two most recent exercises on Career Cup: Given a function rand2 that returns 0 or 1 with equal probability, […]

I like to read a web site called Career Cup, both to enjoy solving some of the programming exercise given there and to find exercise for Programming Praxis. As I write this exercise, here are the two most recent exercises on Career Cup:

- Given a function
`rand2`

that returns 0 or 1 with equal probability, write a function`rand3`

that returns 0, 1 or 2 with equal probability, using only`rand2`

as a source of random numbers. - Given a set of characters and a dictionary of words, find the shortest word in the dictionary that contains all of the characters in the set. In case of a tie, return all the words of the same (shortest) length.

Your task is to write the two programs described above. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

We are delighted to announce GNU Guile release 2.1.3, the next pre-release in what will become the 2.2 stable series.

This release rewrites the ports facility to better support non-blocking concurrent input and output. See the newly rewritten "Input and Output" section of the manual for all details, and the release announcement for a download link.

Here is Sakamoto’s algorithm for calculating the day of the week, taken from the comment that introduces the code: Jan 1st 1 AD is a Monday in Gregorian calendar. So Jan 0th 1 AD is a Sunday [It does not exist technically]. Every 4 years we have a leap year. But xy00 cannot be a […]

Here is Sakamoto’s algorithm for calculating the day of the week, taken from the comment that introduces the code:

Jan 1st 1 AD is a Monday in Gregorian calendar.

So Jan 0th 1 AD is a Sunday [It does not exist technically].Every 4 years we have a leap year. But xy00 cannot be a leap unless xy divides 4 with reminder 0.

y/4 – y/100 + y/400 : this gives the number of leap years from 1AD to the given year. As each year has 365 days (divdes 7 with reminder 1), unless it is a leap year or the date is in Jan or Feb, the day of a given date changes by 1 each year. In other case it increases by 2.

y -= m So y + y/4 – y/100 + y/400 gives the day of Jan 0th (Dec 31st of prev year) of the year. (This gives the reminder with 7 of the number of days passed before the given year began.)

Array t: Number of days passed before the month ‘m+1’ begins.

So t[m-1]+d is the number of days passed in year ‘y’ upto the given date.

(y + y/4 – y/100 + y/400 + t[m-1] + d) % 7 is reminder of the number of days from Jan 0 1AD to the given date which will be the day (0=Sunday,6=Saturday).

int dow(int y, int m, int d) { static int t[] = {0, 3, 2, 5, 0, 3, 5, 1, 4, 6, 2, 4}; y -= m < 3; return (y + y/4 - y/100 + y/400 + t[m-1] + d) % 7; }

Another description is given here.

Your task is to write a program that implements the day-of-week algorithm shown above. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

There's two ways to look at this recent Programming Praxis exercise: implementing a beginner level statistics function or *creating a magical crystal ball that can predict past, present and future*! I chose to approach this problem with the mindset of the latter. Let's make some magic!

The algorithm we're tackling is linear regression. I managed to skip statistics in college (what a shame!), so I don't recall ever being formally taught this technique. Very roughly, if you have the right set of data, you can plot a line through it. You can then use this line to predict values not in the data set.

The exercise gave us this tiny, manufactured, data set:

x y 60 3.1 61 3.6 62 3.8 63 4.0 65 4.1

With linear regression you can answer questions like: what will be the associated value for say, `64`, `66` or `1024` be? Here's my implementation in action:

A few words about the screenshot above. You'll notice that I'm converting my data from a simple list to a generator. A generator in this case is a function that will return a single element in the data set, and returns `'()` when all the data has been exhausted. I chose to use a generator over a simple list because I wanted to allow this solution to scale to large data sets.

Below you'll see a data set that's stored in a file and leverages a file based generator to access its contents. So far, I haven't throw a large data set at this program, but I believe it should scale without issue.

The call to `make-crystal-ball` performs the linear-regression and returns back a function that when provided `x` returns a ~~guess~~ prediction for `y`. What? I'm trying to have a bit of fun here.

Looking around on web I found this example that uses a small, but real data set. In this case, it compares High School and College GPA. Using linear-regression we're able to predict how a High School student with a 2.7, 3.0 or 3.5 GPA is going to do in college. Here's the code:

;; http://onlinestatbook.com/2/regression/intro.html ;; http://onlinestatbook.com/2/case_studies/sat.html (define (gpa-sat-test) (define data (file->generator "gpa-sat-data.scm" (lambda (high-gpa math-sat verb-sat comp-gpa univ-gpa) (list high-gpa univ-gpa)))) (let ((ball (make-crystal-ball data))) (show "2.7 => " (ball 2.7)) (show "3.0 => " (ball 3)) (show "3.5 => " (ball 3.5))))

And the answer is: 2.91, 3.12 and 3.45 respectively. So yeah, that's good news if you were dragging in High School, you're GPA should climb a bit. But High School overachievers should beware, your GPA is most likely to dip. D'oh.

Below is my implementation of this solution. You can also find it on github. As usual I find myself preaching the benefits of Scheme. The code below was written on my Galaxy Note 5 using Termux, emacs and Tinyscheme. With relative ease I was able to implement a generator framework that works for both lists and data files. I'm also able to leverage the Scheme reader so that the data file format is trivial to operate on. Finally, I wrote a generic `sigma` function that walks through my data set once, but performs all the various summations I'll need to calculate the necessary values. In other words, I feel like I've got an elegant solution using little more than lists, lambda functions and sexprs. It's beautiful and should be memory efficient.

Here's the code:

;; https://programmingpraxis.com/2016/06/10/linear-regression/ (define (show . args) (for-each (lambda (arg) (display arg) (display " ")) args) (newline)) (define (as-list x) (if (list? x) x (list x))) (define (g list index) (list-ref list index)) (define (make-list n seed) (if (= n 0) '() (cons seed (make-list (- n 1) seed)))) (define (list->generator lst) (let ((remaining lst)) (lambda () (cond ((null? remaining) '()) (else (let ((x (car remaining))) (set! remaining (cdr remaining)) x)))))) (define (file->generator path scrubber) (let ((port (open-input-file path))) (lambda () (let ((next (read port))) (if (eof-object? next) '() (apply scrubber next)))))) (define (sigma generator . fns) (define (update fns sums data) (let loop ((fns fns) (sums sums) (results '())) (cond ((null? fns) (reverse results)) (else (let ((fn (car fns)) (a (car sums))) (loop (cdr fns) (cdr sums) (cons (+ a (apply fn (as-list data))) results))))))) (let loop ((data (generator)) (sums (make-list (length fns) 0))) (if (null? data) sums (loop (generator) (update fns sums data))))) ;; Magic happens here: ;; m = (n × Σxy − Σx × Σy) ÷ (n × Σx2 − (Σx)2) ;; b = (Σy − m × Σx) ÷ n (define (linear-regression data) (let ((sums (sigma data (lambda (x y) (* x y)) (lambda (x y) x) (lambda (x y) y) (lambda (x y) (* x x)) (lambda (x y) 1)))) (let* ((Sxy (g sums 0)) (Sx (g sums 1)) (Sy (g sums 2)) (Sxx (g sums 3)) (n (g sums 4))) (let* ((m (/ (- (* n Sxy) (* Sx Sy)) (- (* n Sxx) (* Sx Sx)))) (b (/ (- Sy (* m Sx)) n))) (cons m b))))) (define (make-crystal-ball data) (let* ((lr (linear-regression data)) (m (car lr)) (b (cdr lr))) (lambda (x) (+ (* m x) b)))) ;; Playtime (define (test) (define data (list->generator '((60 3.1) (61 3.6) (62 3.8) (63 4.0) (65 4.1)))) (let ((ball (make-crystal-ball data))) (show (ball 64)) (show (ball 66)) (show (ball 1024)))) ;; From: ;; http://onlinestatbook.com/2/regression/intro.html ;; http://onlinestatbook.com/2/case_studies/sat.html (define (gpa-sat-test) (define data (file->generator "gpa-sat-data.scm" (lambda (high-gpa math-sat verb-sat comp-gpa univ-gpa) (list high-gpa univ-gpa)))) (let ((ball (make-crystal-ball data))) (show "2.7 => " (ball 2.7)) (show "3.0 => " (ball 3)) (show "3.5 => " (ball 3.5)))) (gpa-sat-test)

Today’s exercise is in two parts, first a commonly-seen programming exercise and then a variant on it; the origin of the exercise is certainly someone’s homework, but since school is out for the year it doesn’t matter that we do the exercise today. First, write a program that, given an array of integers in unsorted […]

Today’s exercise is in two parts, first a commonly-seen programming exercise and then a variant on it; the origin of the exercise is certainly someone’s homework, but since school is out for the year it doesn’t matter that we do the exercise today.

First, write a program that, given an array of integers in unsorted order, finds the single duplicate number in the array. For instance, given the input [1,2,3,1,4], the correct output is 4.

Second, write a program that, given an array of integers in unsorted order, finds all of the multiple duplicate numbers in the array. For instance, given the input [1,2,3,1,2,4,1], the correct output is [1,2,1].

Your task is to write the two programs that find duplicates in an array. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

Linear regression is a widely-used statistical technique for relating two sets of variables, traditionally called x and y; the goal is to find the line-of-best-fit, y = m x + b, that most closely relates the two sets. The formulas for computing the line of best fit are: m = (n × Σxy − Σx […]

Linear regression is a widely-used statistical technique for relating two sets of variables, traditionally called *x* and *y*; the goal is to find the line-of-best-fit, *y* = *m* *x* + *b*, that most closely relates the two sets. The formulas for computing the line of best fit are:

m= (n× Σxy− Σx× Σy) ÷ (n× Σx^{2}− (Σx)^{2})

b= (Σy−m× Σx) ÷n

You can find those formulas in any statistics textbook. As an example, given the sets of variables

x y 60 3.1 61 3.6 62 3.8 63 4.0 65 4.1

the line of best fit is *y* = 0.1878 *x* − 7.9635, and the estimated value of the missing *x* = 64 is 4.06.

Your task is to write a program that calculates the slope m and intercept b for two sets of variables x and y. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

Christian Goldbach (1690-1764) was a Prussian mathematician and contemporary of Euler. One of the most famous unproven conjectures in number theory is known as Goldbach’s Conjecture, which states that every even number greater than two is the sum of two prime numbers; for example, 28 = 5 + 23. We studied Goldbach’s Conjecture in a […]

Christian Goldbach (1690-1764) was a Prussian mathematician and contemporary of Euler. One of the most famous unproven conjectures in number theory is known as Goldbach’s Conjecture, which states that every even number greater than two is the sum of two prime numbers; for example, 28 = 5 + 23. We studied Goldbach’s Conjecture in a previous exercise.

Although it is not as well known, Goldbach made another conjecture as follows: Every odd number greater than two is the sum of a prime number and twice a square; for instance, 27 = 19 + 2 * (2 ** 2). (The conjecture is sometimes stated as every odd composite number is the sum of a prime number and twice a square, since it is trivially true with 0 as the square root for all prime numbers; alternately, it is sometimes limited so that the number being squared must be positive, in which case there are some odd primes that can be so expressed.) Sadly, it is easy to find a counter-example that disproves Goldbach’s other conjecture.

Your task is to write a program that finds the smallest number that disproves Goldbach’s other conjecture. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

Today’s exercise demonstrates that it is sometimes possible to do a lot with a little. Your task is to write some interesting and useful program in no more than a dozen lines of code. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or […]

Today’s exercise demonstrates that it is sometimes possible to do a lot with a little.

Your task is to write some interesting and useful program in no more than a dozen lines of code. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

It’s fun to learn new programming languages. It’s also useful, even if you never use the new language, because it forces you to think differently about how you do things. Your task is to write a familiar program in an unfamiliar language. When you are finished, you are welcome to read or run ([1], [2]) […]

It’s fun to learn new programming languages. It’s also useful, even if you never use the new language, because it forces you to think differently about how you do things.

Your task is to write a familiar program in an unfamiliar language. When you are finished, you are welcome to read or run ([1], [2]) a suggested solution, or to post your own solution or discuss the exercise in the comments below.

We studied discrete logarithms in two previous exercises. Today we look at a third algorithm for computing discrete algorithms, invented by John Pollard in the mid 1970s. Our presentation follows that in the book Prime Numbers: A Computational Perspective by Richard Crandall and Carl Pomerance, which differs somewhat from other sources. Our goal is to […]

We studied discrete logarithms in two previous exercises. Today we look at a third algorithm for computing discrete algorithms, invented by John Pollard in the mid 1970s. Our presentation follows that in the book *Prime Numbers: A Computational Perspective* by Richard Crandall and Carl Pomerance, which differs somewhat from other sources.

Our goal is to compute *l* (some browsers mess that up; it’s a lower-case ell, for “logarithm”) in the expression *g ^{l}* ≡

- If 0 <
*x*<_{i}*p*/3, then*a*_{i+1}= (*a*+ 1) mod (_{i}*p*− 1),*b*_{i+1}=*b*, and_{i}*x*_{i+1}=*t x*(mod_{i}*p*). - If
*p*/3 <*x*< 2_{i}*p*/3, then*a*_{i+1}= 2*a*mod (_{i}*p*− 1),*b*_{i+1}= 2*b*mod (_{i}*p*− 1), and*x*_{i+1}=*x*_{i}^{2}mod*p*. - If 2
*p*/3 <*x*<_{i}*p*, then*a*_{i+1}=*a*,_{i}*b*_{i+1}= (*b*+ 1) mod (_{i}*p*− 1), and*x*_{i+1}=*g x*mod_{i}*p*.

Splitting the computation into three pieces “randomizes” the calculation, since the interval in which *x _{i}* is found has nothing to do with the logarithm. The sequences are computed until some

Thus, Pollard’s rho algorithm consists of iterating the sequences until a match is found, for which we use Floyd’s cycle-finding algorithm, just as in Pollard’s rho algorithm for factoring integers. Here are outlines of the two algorithms, shown side-by-side to highlight the similarities:

# find d such that d | n # find l such that g**l = t (mod p) function factor(n) function dlog(g, t, p) func f(x) := (x*x+c) % n func f(x,a,b) := ... as above ... t, h, d := 1, 1, 1 j := (1,0,0); k := f(1,0,0) while d == 1 while j.x <> k.x t = f(t) j(x,a,b) := f(j.x, j.a, j.b) h = f(f(h)) k(x,a,b) := f(f(k.x, k.a, k.b)) d = gcd(t-h, n) d := gcd(j.a-k.a, p-1) return d return l ... as above ...

Please pardon some abuse of notation; I hope the intent is clear. In the factoring algorithm, it is possible that *d* is the trivial factor *n*, in which case you must try again with a different constant in the *f* function; the logarithm function has no such possibility. Most of the time consumed in the computation is the modular multiplications in the calculations of the *x* sequence; the algorithm itself is O(sqrt *p*), the same as the baby-steps, giant-steps algorithm of a previous exercise, but the space requirement is only a small constant, rather than the O(sqrt *p*) space required of the previous algorithm. In practice, the random split is made into more than 3 pieces, which complicates the code but speeds the computation, as much as a 25% improvement on average.

Your task is to write a program that computes discrete logarithms using Pollard’s rho algorithm. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

The high school two blocks from me just had their annual picnic, my youngest daughter just graduated from college, and my primarily academic readership suddenly dropped in half (history suggest it will stay low until mid-August), so it seems to be the right season to have a simple data-processing task involving student test scores. Given […]

The high school two blocks from me just had their annual picnic, my youngest daughter just graduated from college, and my primarily academic readership suddenly dropped in half (history suggest it will stay low until mid-August), so it seems to be the right season to have a simple data-processing task involving student test scores.

Given a list of student names and test scores, compute the average of the top five scores for each student. You may assume each student has as least five scores.

Your task is to compute student scores as described above. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

I’ve been busy at work and haven’t had time to prepare an exercise for today. I apologize. Your task is to solve a previous exercise that you haven’t yet solved. Have fun!