Homework 5
Monads
This homework is written in literate Haskell; you can download the raw source to fill in yourself. You’re welcome to submit literate Haskell yourself, or to start fresh in a new file, literate or not.
Please submit homeworks via the new submission page.
This homework (and all the ones after it), you are free to work on your own or in groups of up to three. Please make sure that your whole group—and only your group—are listed as collaborators on any submission.
In this homework, you’ll be working with monads in a variety of ways: to speed up a shuffling algorithm; to create a generic framework for MapReduce; and to generate random values for testing.
I’ve imported the libraries you’ll need. Look at them before you start! You’ll want to use both the Haskell language documentation as well as Hackage and Hoogle.
If you’re running Haskell on your own computer and you installed the Haskell stack, you should be able to install the two libraries we need by running at the command line stack install random QuickCheck. These libraries should already be installed on the lab machines, but they may not be; the same stack install random QuickCheck command should work.
You are of course allowed to import other libraries. It may even make your solutions easier!
module Hw05 where
import Control.Monad
import Data.Array.IO
import qualified Data.Map as Map
import Data.Map (Map,(!))
import System.Environment
import System.Exit
import System.IO
import System.Random
import Test.QuickCheck hiding (shuffle)Problem (1): shuffling
In this problem, we’re going to “shuffle” a list, reordering it randomly.
To start, let’s get familiar with System.Random.
Write a function that takes two numbers, low and high, and returns a random number n such that low <= n <= high, i.e., inclusively within the range.
Now write a function that takes a list and shuffles it. The straightforward algorithm is O(n2):
- Given a non-empty list
xs, - randomly split the list into an element
yand the rest of the listys, - cons
yonto the shuffling ofys.
Don’t worry, we’ll speed it up in a minute.
Don’t forget that you can run :set +s to get timing information in GHCi. My implementation on my computer runs sum <$> shuffleList [0..10000] 3.26 seconds.
It turns out that there’s a much faster, O(n) algorithm for shuffling: the Fisher–Yates shuffle. It works on arrays, not linked lists, so we’ll have to use Haskell’s arrays.
Haskell’s arrays are a little funny: arrays are parameterized by two things: the type of their index and the monad in which they’ll be used. We’ll work with IOArrays. The IOArray type represents arrays that can be used in the IO monad. We’ll interact with these arrays using the MArray interface.
Let’s take a brief look at IOArray. It has kind * -> * -> *. The first type it needs is the type of its indices… we can just use Int for that, but it’s interesting that we can use any type in the Ix class. The second type it needs is the type of its contents. Shuffling won’t care about that, so we’ll end up working with IOArray Int a.
As a warmup, write a function that takes a list and generates a corresponding array. It’s worth noting that the bounds that Haskell uses in, e.g., newListArray are inclusive, per Data.Ix.
Okay: let’s do it. Implement the Fisher–Yates shuffling algorithm that takes a given array and shuffles it.
Now use your array-based function shuffle to work on lists. Be sure to test your code on a wide variety of inputs!
My version of fastShuffle runs much more quickly than the naive one: sum <$> fastShuffle [0..10000] runs in 0.04 seconds!
Finally, write a function that reads in a file and shuffles the order of its lines. You should ignore lines with no characters on them and your final output should not end in a newline.
Finally, this shuffling program is useful enough that we should make it a command. To do so, you’ll need to write a main function, which should have type IO ().
First and foremost, to compile your program, you’ll run a command like: stack ghc -- Hw05.lhs -main-is Hw05.main -o shuffle. This will compile your program to an executable called shuffle. (The -main-is flag is necessary because the default in Haskell is to have a file named Main.hs.)
How should your program behave? It should take a single, optional argument indicating a filename to shuffle. Without an argument (or with an argument of -), it should read the content to be sorted from standard input. If more than one argument is given, you should print a “usage” message on standard error and exit with a non-zero exit code.
Either way, your program should read the lines of the input (file or standard input), shuffle them, and then print out the shuffled input.
Look at the various System modules to find out how to parse command-line arguments.
Problem (2): monadic MapReduce
MapReduce is a model for data parallel computation. We’ll look at data parallel programming after Thanksgiving, but for now let’s try to understand MapReduce as is.
Our mappers will take an input of type a and produce a list of key-value pairs, where keys have type k and values have type v.
Our reducers take a key and list of values and produces a new (ideally shorter!) list of values.
The actual MapReduce implementaiton has three real phases: mapping, shuffling, and reducing. Here’s a simple implementation:
mapReduce :: Ord k => Mapper a k v -> Reducer k v -> [a] -> [(k,[v])]
mapReduce m r = reduce r . shuffleKeys . concatMap (map listifyVal . m)
where listifyVal (k,v) = (k,[v])
shuffleKeys = Map.fromListWith (++)
reduce r = Map.toList . Map.mapWithKey rThe canonical MapReduce example is word count. (Riveting, isn’t it?) Here’s how to implement word count in MapReduce: given a list of documents, a mapper breaks a given document into its consituent words, where appearance of a word maps to 1. After shuffling, identical words will be grouped together; the reducer sums up each token.
wordCount = mapReduce countWords sumCounts
where countWords = map (\w -> (w,1)) . words
sumCounts _ cs = [sum cs]I’m not sure why Google is so proud of what amounts to eight lines of code and a slow way to count words.
Let’s modify the MapReduce paradigm to allow for mappers and reducers that generate monadic computations, like so:
Note that a MapperM returns its list of key-value pairs inside of some monad m; ReducerM is similar.
Adapt mapReduce above to define mapReduceM:
mapReduceM :: (Ord k, Monad m) => MapperM m a k v -> ReducerM m k v -> [a] -> m [(k,[v])]
mapReduceM m r input = undefinedTo test, here’s an adaptation of the wordCount example above.
wordCountM = mapReduceM countWords sumCounts
where countWords = return . map (\w -> (w,1)) . words
sumCounts w cs = do
when (length cs > 1) $ putStrLn $ "Lots of " ++ w ++ "!"
return [sum cs]If you’re having trouble, I recommend piecing apart the function and relying on types to help you get through. If you’re truly stuck, a good place to start is refactoring mapReduce so it doesn’t use composition ((.)).
Problem (3): QuickCheck
We’ll be using QuickCheck to write some tests.
Write a QuickCheck property to check that reverse is involutive, i.e., that reversing a reversed list yields the original list.
Write a QuickCheck property to check that checks the Collatz conjecture for a given number greater than 0.
Write a QuickCheck property that expresses the correctness of your fastShuffle function. No need to go for full correctness of every potential property of your shuffle, e.g., that it’s pseudorandom. You might need to write a type signature. Check out Test.QuickCheck.Monadic.
data ArithExp =
Num Int
| Plus ArithExp ArithExp
| Times ArithExp ArithExp
| Neg ArithExp
deriving Show
eval :: ArithExp -> Int
eval (Num i) = i
eval (Plus e1 e2) = eval e1 + eval e2
eval (Times e1 e2) = eval e1 * eval e2
eval (Neg e) = 0 - eval eWrite a generator that generates arbitrary ArithExps. Use it to define an Arbitrary instance for ArithExp… keep in mind that we don’t want to generate giant data structures, so you may need to keep track of sizes.
Write a test to ensure that Plus e e behaves the same as Times 2 e for all expressions e.