The essence of live coding

Change the program, keep the state! https://github.com/turion/essence-of-live-coding

Manuel Bärenz

Introduction

The essence of live coding

Live coding framework prototype
Capture the essential idea
Audio, video, web, …
Modular FRP DSL
Type-driven, automatic, extensible state migration

Change the program, keep the state!

How does it work?

A live program

Internal state
State transition function
Side effects

data LiveProgram m = forall s .
  LiveProgram
  { liveState :: s
  , liveStep  :: s -> m s
  }

runLiveProgram :: LiveProgram m -> m ()
runLiveProgram LiveProgram { .. } = do
  liveState' <- liveStep liveState
  runLiveProgram LiveProgram { liveState = liveState', .. }

Polymorphic in m:

Determinism (no hidden IO)
Arbitrary effects: Bind to any backend

Example

data State = State
  { nVisitors :: Integer
  }

server greeting = LiveProgram { .. } where
  liveState = State 1
  liveStep State { .. } = do
    putStrLn $ greeting ++ show nVisitors
    return State { nVisitors = nVisitors + 1, .. }

yeOldeServer = server "Greetings, number "

Change the program…

nuSrvr = server "HI #"

Greetings, number 1
Greetings, number 2
HI #3

Thank you for your attention.

That was a joke.

…change the program…

data State = State
  { nVisitors :: Integer
  , lastTime  :: UTCTime
  }

server greeting = LiveProgram { .. } where
  liveState = State 1 $ read "2019-07-24 19:00:00 UTC"
  liveStep State { .. } = do
    putStrLn $ greeting ++ show nVisitors
    lastTime <- getCurrentTime
    return State { nVisitors = nVisitors + 1, .. }

…but the new state won’t type-check!

…keep the state??

Well-known problem in databases
Need state migrations
Should be automatic

Example

Migrating to new state should…

…preserve nVisitors
…insert initial/default value for lastTime

⇒ Need old live state and new initial state

migrate :: sNew -> sOld -> sNew

Actual implementation: Generics

migrate :: (Data sNew, Data sOld) => sNew -> sOld -> sNew
migrate = ... -- Just some 50 lines generic code

Conversion from/to…

…same record fields
…same constructor names
…newtypes

Type-driven migration

Small restriction: Add Data to all states:

data LiveProgram m = forall s .
  Data s =>
  LiveProgram
  { liveState :: s
  , liveStep  :: s -> m s
  }

The migration is derived from the state type!

Functional Reactive Programming

What’s missing? Modularity

Want to combine new live programs from existing building blocks.
“Functional”

⇒ Add inputs and outputs to live programs!

Cells

data Cell m a b = forall s . Data s => Cell
  { cellState :: s
  , cellStep  :: s -> a -> m (b, s)
  }

`Category`

CategoryId CategoryCompose

id :: Cell m a a

(>>>)
  :: Monad m
  => Cell  m a b
  -> Cell  m   b c
  -> Cell  m a   c

`Arrow`

arr
  ::       (a -> b)
  -> Cell m a    b

(***)
  :: Monad m
  => Cell  m  a      b
  -> Cell  m     c      d
  -> Cell  m (a, c) (b, d)

Composing cells to live programs

sensor    :: Cell IO () a
signalFun :: Cell m     a b
actuator  :: Cell IO      b ()

liveCell :: Cell m () () -> LiveProgram m

mainProgram :: LiveProgram IO
mainProgram = liveCell
  $ sensor >>> signalFun >>> actuator

Wait… I’ve seen this before!?

data MSF m a b => MSF { unMSF :: a -> m (b, MSF m a b) }

Haskell Symposium 2016

Example

sumC = Cell { .. } where
  cellState = 0
  cellStep accum a = return (accum, accum + a)

stepRate = 30
integrate = arr (/ stepRate) >>> sumC

Arrow notation

freeFall
  =   arr (const (0, -9.81))
  >>> (   (integrate >>> integrate)
      *** (integrate >>> integrate))

freeFall = proc () -> do
  let (accX, accY) = (0, -9.81)
  velX <- integrate -< accX
  posX <- integrate -< velX
  velY <- integrate -< accY
  posY <- integrate -< velY
  returnA           -< (posX, posY)

More Arrow perks

ArrowChoice: (Transient) control flow
ArrowLoop: Recursion (but beware!)

  let (accX, accY) = (0, -9.81)
  velX <- integrate -< accX
  posX <- integrate -< velX
  rec
    velY <- if posY >= 0
      then do
        integrate -< accY
      else
        returnA   -< 0
    posY <- integrate -< velY
  returnA           -< (posX, posY)

Gloss backend

PictureM: Retrieve events, IO, add pictures

glossCell :: Cell PictureM () ()
glossCell = proc () -> do
  events <- constM ask  -< ()
  arrM $ liftIO . print -< events
  ball <- ballSim       -< events
  addPicture            -< ballPic ball

HandlingStateT: Handle Gloss thread

glossRunCell :: Cell (HandlingStateT IO) () ()
glossRunCell = glossWrapC glossSettings glossCell

Monadic control flow

Throwing and catching exceptions

wait :: Monad m => Double -> Cell (ExceptT () m) a a
wait tMax = proc a -> do
  t <- integrate -< 1
  if t >= tMax
    then throwC  -< ()
    else returnA -< a

afterTwoSeconds :: Cell m a String
afterTwoSeconds = safely $ do
  try  $   arr (const "Waiting...")
       >>> wait 2
  safe $   arr (const "Done waiting.")

Extra perks

GHCi runtime

Custom GHCi commands:

Launch live program in separate thread
Edit file, reload, while keeping program running
(uses package foreign-store under the hood)
Gloss and Pulseaudio adapter

QuickCheck

Test output of cells for arbitrary series of inputs
Launch test cell before migrating “production” cell
Data ⇒ automatically generate cell state for tests

Debugging

“A debugger is a live program that can read and modify the state of another live program.”

newtype Debugger m = Debugger
  { getDebugger :: forall s .
      Data s => LiveProgram (StateT s m)
  }

withDebugger
  :: Monad       m
  => LiveProgram m
  -> Debugger    m
  -> LiveProgram m

Example: Gloss debugger

Outlook

What I haven’t shown

Works for web servers as well
Extensible by custom user migrations
Integrate into external main loops
Control flow: The details

Further directions

Backends (Audio, web, OpenGL, …)
Integrate with asynchronous FRP framework (Rhine. ICFP 2019. Iván Pérez & MB)
Ensuring state properties beyond Haskell types (LiquidHaskell)

Thanks! Questions?

Manuel Bärenz	Essence Of Live Coding
Github	https://github.com/turion/essence-of-live-coding
Hackage	https://hackage.haskell.org/package/essence-of-live-coding
Article	https://www.manuelbaerenz.de/essence-of-live-coding/EssenceOfLiveCoding.pdf
Presentation	https://www.manuelbaerenz.de/essence-of-live-coding/EssenceOfLiveCodingPresentation.html
Tutorial	https://github.com/turion/essence-of-live-coding-tutorial/
Abstract	https://www.manuelbaerenz.de/essence-of-live-coding/EssenceOfLiveCodingAbstract.pdf
Appendix	https://www.manuelbaerenz.de/essence-of-live-coding/EssenceOfLiveCodingAppendix.pdf

Backup

Migrations

data Migration = Migration
  { runMigration :: forall a b . (Data a, Data b)
      => a -> b -> Maybe a
  }

instance Monoid Migration where ...

migrateWith
  :: (Data a, Data b)
  => Migration
  -> a -> b -> a

QuickCheck

instance (Arbitrary a, Show a, Testable prop)
  => Testable (Cell IO a prop) where ...

testCell :: Monad m => Cell m (Positive Int) Bool
testCell = arr getPositive >>> sumC >>> arr (>= 0)

 > quickCheck testCell
+++ OK, passed 100 tests.

logTest :: Monad m => Cell m a prop -> Cell (WriterT [prop] m) a ()
logTest cell = liftCell cell >>> arrM (return >>> tell)

liveCheck :: Testable prop => Bool
  -> LiveProgram (WriterT [prop] IO)
  -> LiveProgram                 IO

FRP internal state

data Composition state1 state2 = Composition
  { state1 :: state1
  , state2 :: state2
  } deriving Data

instance Monad m => Category (Cell m) where
  id = ...

  Cell state1 step1 >>> Cell state2 step2 = Cell { .. }
    where
      cellState = Composition state1 state2
      cellStep ... = ...

Exceptions

runExceptT :: ExceptT e m b -> m (Either e b)

runExceptC
  :: (Data e, Monad m)
  => Cell (ExceptT e m) a           b
  -> Cell            m  a (Either e b)

data ExceptState state e
  = NotThrown state
  | Exception e
  deriving Data

Finite exceptions

(>>>=) :: (Data e1, Monad m)
  => Cell (ExceptT e1    m)      a  b
  -> Cell (ExceptT    e2 m) (e1, a) b
  -> Cell (ExceptT    e2 m)      a  b

class Finite e where
  commute
    :: Monad      m
    => (e -> Cell m     a  b)
    -> Cell       m (e, a) b

Coalgebras

type StateTransition m a b s = a -> m (b, s)

data MSF m a b = MSF (StateTransition m a b (MSF m a b))

data Coalg m a b = forall s .
     Coalg s (s -> StateTransition m a b s)

cellCoalgebra :: Cell m a b -> Coalg m a b
coalgebra     :: MSF  m a b -> Coalg m a b

finality      :: Monad m => Coalg m a b -> MSF m a b
finalityC     :: Monad m => Cell  m a b -> MSF m a b

coalgebraC    :: Data (MSF m a b) => MSF m a b -> Cell m a b