Goblins has a secure persistence mechanism that allows for serializing
an entire live object graph.  This mechanism allows objects to
describe themselves with @emph{only} the capabilities they have,
preventing malicious objects from being restored with more power than
they had when they were saved.

If all objects in a graph can be serialized then they can then be
stored (e.g. to disk) and later restored (or ``rehydrated'') back into
live objects that retain all of their original relationships.  Due to
the nature of how the persistence mechanism works, it also can be used
as a powerful tool for object upgrades and live hacking.

@menu
* Persistence tutorial::
* Persistence capable objects::
* Operations on actormaps::
* Persistent vats::
* Persistence environments::
* Upgrading through persistence::
* Persistence stores::
* Sleepy actors::
@end menu

@node Persistence tutorial
@section Persistence tutorial

When writing Goblins code without considering persistence, we simply
define an outer procedure (the constructor, usually
@code{^my-constructor}) and an inner procedure (the behavior, most
often using an anonymous @code{lambda}) that's called when the object
receives a message.  Persistence capable objects are a little
different; while they still follow this general pattern, they also
need to provide a way to @emph{describe} themselves.  Fortunately,
Goblins provides the @code{define-actor} syntax which makes this
easier.  For more advanced cases where @code{define-actor} is
unsuitable, see @ref{Persistence capable objects} for the lower-level
details of creating persistent objects.

Let's start with the cell example from the @ref{Tutorial}:

@lisp
(define* (^cell bcom #:optional (val #f))
  (case-lambda
    (()         ; get
     val)
    ((new-val)  ; set
     (bcom (^cell bcom new-val)))))
@end lisp

To make @code{^cell} persistence capable, @code{define} can be changed
to @code{define-actor}:

@lisp
(use-modules (goblins))

(define-actor (^cell bcom #:optional (val #f))
  (case-lambda
    (()         ; get
     val)
    ((new-val)  ; set
     (bcom (^cell bcom new-val)))))
@end lisp

And it's done! Now, let's try it out.  To use persistence, there's an
additional prerequisite: a @emph{persistence environment} must be
defined.  ``What is this extra thing?'', I hear you ask.

In short, serializing an object graph requires translating each object
from a Scheme procedure representing its current behavior (which
cannot be serialized) to a serializable @emph{self-portrait}.  Goblins
does this by storing each object as a mapping of labels to
self-portraits.  In the @code{^cell} constructor above,
@code{define-actor} has taken care of the self-portrait for us.

To restore an object graph from its serialized form, a
@emph{persistence environment} is required.  A persistence environment
is simply a mapping between labels and constructor procedures.  To
demonstrate, let's define a simple persistence environment that only
has our shiny new persistent @code{^cell} in it:

@lisp
(define my-env
  (make-persistence-env
   `((((my-cool-scheme-project cell) ^cell) ,^cell))))

;;    |-----------------------------------| |----|
;;                    label                  proc
@end lisp

Just remember that each label @emph{must} be unique within the
environment, or else you'll get into quite the pickle (for more
information, see @ref{Persistence environments}).

The persistence tools work with both @ref{Vats} and the lower-level
@ref{Actormaps} (since vats are built on top of actormaps).
Persistent vats require a store: a place for the vat to read and write
serialized object graph data.  One such store is the @ref{Memory
store}, which is @emph{ephemeral} and not actually saved to any kind
of long-term storage.  A persistent vat with a memory store can be
spawned like this:

@lisp
(use-modules (goblins)
             (goblins persistence-store memory))

(define memory-store (make-memory-store))
(define-values (my-vat my-cell)
  (spawn-persistent-vat
    my-env
    (lambda ()
      (spawn ^cell))
    memory-store))
@end lisp

Hold on, did we just spawn a @code{^cell} object while spawning the
vat?

Yes, we did!  Notably, this is not necessary for all the objects in
the graph, just the @emph{roots} (the objects that spawn all the other
objects).  When @code{spawn-persistent-vat} is called it first checks
in the store to see if there's any saved data.  If not then the thunk
(the @code{lambda} with no arguments above) is called to spawn the
root objects for the first time.  If there is saved data then it is
deserialized and the associated persistence environment is used to
restore the object graph.

Since the store is empty initially, the value of @code{my-cell} is a
reference to a freshly spawned cell:

@lisp
> ,enter-vat my-vat
goblins[1]> ($ my-cell 'apple)
@end lisp

Now that the persistent vat and its roots have spawned, the object
graph has @emph{automatically} been saved to the store.  By default, a
persistent vat will continuously update the store upon changes to
object state.

When debugging it can be useful to see the serialized form of an
object.  To do this from the REPL, use the
@command{vat-object-portrait} metacommand:

@lisp
goblins[1]> ,vat-object-portrait my-cell
(apple)
@end lisp

Now, let's look at a more advanced example of a persistent object.
We'll first build it without persistence in mind and then see some of
the problems we might encounter when making an object persistable.
Below is an object representing a player ship in a space shooter video
game.

@lisp
(use-modules (goblins actor-lib methods))

(define (^player-ship bcom init-x init-y)
  (define x (spawn ^cell init-x))
  (define y (spawn ^cell init-y))
  (define (dead-beh)
    (error "I'm already dead!"))
  (define active-beh
    (methods
     ((x) ($ x))
     ((y) ($ y))
     ((fire) "pew pew")
     ((kill) (bcom dead-beh))
     ((tick) ($ x (1+ ($ x))))))
  active-beh)
@end lisp

This ship object contains the (x, y) coordinates for its current
position.  These coordinates are stored in cells so that the position
can be changed over time.  The ship spawns in the ``active'' state and
responds to several methods:

@itemize

@item @code{x} and @code{y}
Get the current position.

@item @code{fire}
Shoot, or rather just say ``pew pew''.

@item @code{kill}
Transitions the object to the ``dead'' state.

@item @code{tick}
Move the forward by 1 unit on the X axis.  This method could be hooked
up to a game loop to animate the player.

@end itemize

To make the ship persist, we could try simply changing @code{define}
to @code{define-actor}:

@lisp
(define-actor (^player-ship bcom init-x init-y)
  (define x (spawn ^cell init-x))
  (define y (spawn ^cell init-y))
  (define (dead-beh)
    (error "I'm already dead!"))
  (define active-beh
    (methods
     ((x) ($ x))
     ((y) ($ y))
     ((fire) "pew pew")
     ((kill) (bcom dead-beh))
     ((tick) ($ x (1+ ($ x))))))
  active-beh)
@end lisp

At first it might seem like that was enough, but there are actually
some big problems when we try it out:

@lisp
goblins[1]> (define ship (spawn ^player-ship 0 0))
goblins[1]> ,vat-object-portrait ship
(0 0)
@end lisp

So far, so good.  Now let's try ticking the game loop to see if the X
coordinate changes:

@lisp
goblins[1]> ($ ship 'tick)
goblins[1]> ,vat-object-portrait ship
(0 0)
@end lisp

Hmm, that's not right.  Well, what happens if we kill the player?

@lisp
goblins[1]> ($ ship 'kill)
goblins[1]> ,vat-object-portrait ship
(0 0)
@end lisp

Okay, that's the same as before.  Doesn't seem like the object would
be restored in the dead state.  What's happening?

The problem is that the @code{^player-ship} constructor takes two
arguments (the initial X and Y coordinates) but then spawns cells
internally.  These cells live within the @emph{closure} of the object
and are thus inaccessible to the persistence system (and anything
else, really).  We need a constructor that accepts all of the values
that are needed to serialize a complete self-portrait.  The new player
ship definition below does exactly that:

@lisp
(define-actor (^player-ship bcom x y #:key (active? #t))
  (define (dead-beh)
    (error "I'm already dead!"))
  (define active-beh
    (methods
     ((x) x)
     ((y) y)
     ((fire) "pew pew")
     ((kill) (bcom (^player-ship bcom x y #:active? #f)))
     ((tick) (bcom (^player-ship bcom (1+ x) y)))))
  (if active?
      active-beh
      dead-beh))
@end lisp

Let's test it out:

@lisp
goblins[1]> (define ship (spawn ^player-ship 0 0))
goblins[1]> ,vat-object-portrait ship
(0 0 #:active? #t)
@end lisp

Looks good so far.  Now let's try ticking it:

@lisp
goblins[1]> ($ ship 'tick)
goblins[1]> ,vat-object-portrait ship
(1 0 #:active? #t)
@end lisp

Great!  Finally, let's make sure we can kill it:

@lisp
goblins[1]> ($ ship 'kill)
goblins[1]> ,vat-object-portrait ship
(1 0 #:active? #f)
@end lisp

Everything works!

Instead of mutating internal cells, we simply pass updated values to
the constructor using @code{bcom} and now @code{define-actor}'s
built-in self-portrait contains all of the necessary data to restore
the player ship later.  In summary, when using @code{define-actor} we
need to think of both the behavior and state of an object as being
determined by the constructor rather than relying upon internal state.

There are many other ways to implement this object.  Let's look at two
more and then put these different ideas together to create a much more
advanced player ship.

First, we don't have to do away with cells.  We could pass cells into
the constructor directly, but then we would lose some of the
ergonomics from the previous version of the constructor; users would
have to manually spawn these cells.  Here's a solution that continues
to use cells but keeps the old interface:

@lisp
(use-modules (goblins actor-lib cell))

;; IMPORTANT: note this is now ^player-ship* not ^player-ship.
(define-actor (^player-ship* bcom x y #:key (active? #t))
  (define (dead-beh)
    (error "I'm already dead!"))
  (define active-beh
    (methods
     ((x) ($ x))
     ((y) ($ y))
     ((fire) "pew pew")
     ((kill) (bcom (^player-ship bcom x y #:active? #f)))
     ((tick) ($ x (1+ ($ x))))))
  (if active?
      active-beh
      dead-beh))

(define (^player-ship bcom x y)
  (spawn ^player-ship*
         (spawn ^cell x)
         (spawn ^cell y)))

;; NOTE: Because of the feature of returning an object described
;; below, only ^player-ship* needs to be in the environment.
(define my-env
  (make-persistence-env
    `((((game) ^player-ship) ,^player-ship*))
    #:extends cell-env))
@end lisp

This wrapper pattern involves a public constructor with the desired
interface that does some setup and then calls a private constructor.
In this case, the public interface both prevents a user from spawning
a dead ship and handles the ``cellification'' of the X and Y
coordinates.

A typical constructor returns a procedure that is used as the object's
initial behavior.  However, it's also possible to return a reference
to another object!  Instead of creating a new object, Goblins will
simply return that reference to the caller of the constructor.

Despite @code{^player-ship} being a wrapper around
@code{^player-ship*}, both procedures are defined at the top-level.
In fact, this pattern @emph{would not work} if @code{^player-ship*}
was encapsulated within @code{^player-ship}.  This is because the
persistence environment needs access to the private constructor for
the purpose of restoration.

The new persistence environment above references only the private
constructor, @code{^player-ship*}.  This is because, as explained
above, the @code{^player-ship} is not a true constructor; it merely
delegates to @code{^player-ship*}.  Note also that, instead of using
the version of @code{^cell} we defined above, the example uses the one
from the @code{(goblins actor-lib cell)} module.  So, our persistence
environments needs to @emph{extend} @code{cell-env} provided by that
module.  Persistence environments can be composed together.

Onto the second variant.  Having a sort of ``state'' flag passed into
the constructor allows the ship to decide if the active or dead
behavior should be used, but there are other ways to do this.  Let's
look at a particularly interesting technique which uses the
@ref{Swappable} module, @code{(goblins actor-lib swappable)}:

@lisp
(use-modules (goblins actor-lib swappable))

;; Public constructor again, except this time spawning the swappable
;; object as well as cellifying the x and y coordinates.
(define (^player-ship bcom x y)
  (define-values (beh-proxy beh-swapper)
    ;; NOTE: Due to how swappable works, it must be spawned with an
    ;; initial reference. We'll use a placeholder one which we will
    ;; immediately switch away from.
    (swappable (spawn (lambda _ (lambda _ 'noop)))))
  ;; Swap away from the dummy placeholder above to an alive ship.
  ($ beh-swapper (spawn ^player-ship/alive
      (spawn ^cell x)
      (spawn ^cell y)
      beh-swapper))
  ;; Return the proxy.
  beh-proxy)

;; The behavior and state for the alive player ship.
(define-actor (^player-ship/alive bcom x y swapper)
  (methods
   ((tick) ($ x (1+ ($ x))))
   ((x) ($ x))
   ((y) ($ y))
   ((fire) "pew pew")
   ((kill) ($ swapper (spawn ^player-ship/dead x y swapper)))))

;; The behavior and state for the dead player ship.
(define-actor (^player-ship/dead bcom x y swapper #:optional (countdown 30))
  (methods
   ((tick)
    ;; When dead instead of moving, just wait until the countdown
    ;; has reached zero and then swap to alive.
    (if (zero? countdown)
        ($ swapper (spawn ^player-ship/alive x y swapper))
        (bcom (^player-ship/dead bcom x y swapper (1- countdown)))))
   ((x) ($ x))
   ((y) ($ y))
   ((fire) 'noop)   ; cannot fire when dead
   ((kill) 'noop))) ; cannot be killed when dead

;; Note because the swappable and cell objects are being passed in and
;; thus need to be persisted in the graph, we *MUST* include those in
;; persistence environment.
(define game-env
  (make-persistence-env
   `((((game) ^player-ship/alive) ,^player-ship/alive)
     (((game) ^player-ship/dead) ,^player-ship/dead))
   #:extends (list swappable-env cell-env)))
@end lisp

In the above example, the @code{^player-ship} constructor is returning
the @emph{proxy} from @code{swappable} as the reference for the player
ship.  The proxy will forward messages to the object representing the
ship's current state.  The capability to swap the state of the ship is
passed to the @code{^player-ship/alive} and @code{^player-ship/dead}
constructors.  While this is probably overkill for our player ship,
this example illustrates that @code{swappable} can be a powerful way
to implement an object with many different possible behaviors.

It may be tempting to simplify this by removing the swapper and using
@code{bcom} like @code{(bcom (spawn ^player-ship/dead ...))}, but this
@emph{does not} work.  @code{bcom} accepts a procedure representing
the new behavior, not an object reference.  In other words, Goblins
does not allow an object to become something that it's not (that would
be a security issue).

And that's it for the tutorial!  Now, go out and save the world!

@node Persistence capable objects
@section Persistence capable objects

When using the persistence mechanism you take a portrait of a graph, the
graph has one or more roots. All object references starting from those
root objects will be serialized, this even for graphs which contain
cycle. Take the following graph:

@verbatim

       A           H                 <-- roots
      /|\          |
     / E \         J
    B     F
   / \
  C   D
@end verbatim

The root objects would be objects @code{A} and @code{H}.

In order for an object to have persistence support it must be able to
create a self portrait of itself. The self portrait is a collection of
all the data and capabilities an object has thats needed to restore
itself. This can be done by creating a function will be called upon
taking a portrait return a list containing all the portrait data. The
data can include references to other local objects.

Lets take a really simple example which is just an object which has a
name it's known by and greets you:

@lisp
(use-modules (goblins))

(define (^greeter _bcom our-name)
  (lambda (their-name)
    (format #f "Hello ~a, my name is ~a" their-name
            our-name)))
@end lisp

This object is not persistence capable, that is it does not currently
have a mechanism to describe itself to create the self portrait. Trying
to take a portrait of a graph with a non-persistence aware object will
result in an error. The easiest way to convert this to a persistence
aware object is to use the @code{define-actor} macro:

@lisp
(use-modules (goblins)
             (goblins actor-lib define-actor))

(define-actor (^greeter _bcom our-name)
  (lambda (their-name)
    (format #f "Hello ~a, my name is ~a" their-name
            our-name)))
@end lisp

That's it! The @code{define-actor} macro is probably good enough to
handle most use cases. There will however be cases where you need to be
a bit more explicit regarding serialization. The @code{define-actor}
macro takes all of the values it's spawned with, and keeps them around
so that you can be re-spawned with those values, however there may be
times for example where you need to convert certain values to a
serializable form.

In cases where you do need to be more explicit and can't rely on the
macro, you can do this manually. To do this we can define a
self-portrait function which is called when the persistence system is
taking a portrait, this function is expected to return either a list of
values or a versioned tagged list of values (more on versioning
@xref{Upgrading through persistence}). Here's the same greeter but
manually specifying the self-portrait behavior:

@lisp
(use-modules (goblins))

(define (^greeter _bcom our-name)
  (define (main-beh their-name)
    (format #f "Hello ~a, my name is ~a" their-name
            our-name))
  (define (self-portrait)
    (list our-name))
  (portraitize main-beh self-portrait))
@end lisp

The persistence system supports the following types:

@itemize @bullet
@item
number
@item
boolean
@item
character
@item
string
@item
symbol
@item
bytevector
@item
list
@item
vector
@item
keyword
@item
ghash
@item
tagged value (from the @code{(goblins abstract-types)} module)
@item
zilch (from the @code{(goblins abstract-types)} module)
@item
local object refr
@item
resolved local promises
@item
All OCapN IDs from @code{(goblins ocapn ids)}
@end itemize

Any other value of another type needs to be converted into one of the
supported types above.

As mentioned above, the goblins persistence system can handle references
to objects within the same actormap or vat, you simply include the refer
when persisting. Let's take a look at a greeter which does that, keeping
track of the number of types its been called:

@lisp
(use-modules (goblins)
             (goblins actor-lib cell))

(define-actor (^greeter* _bcom our-name number-of-times)
  (lambda (their-name)
    (let ((times ($ number-of-times)))
      ($ number-of-times (+ 1 times))
      (format #f "Hello ~a, my name is ~a (called ~a)"
              their-name our-name times))))

(define* (^greeter bcom our-name #:optional [number-of-times 0])
  (spawn ^greeter* our-name (spawn ^cell number-of-times)))
@end lisp

This example uses @code{define-actor} with a separate constructor which
``cellifies` the number-of-times and has a default argument. The
@code{greeter} constructor is the one used to spawn the object, it
itself spawns another object returning the refr rather than behavior.
@code{define-actor} will include both the name (a string, like before)
and a local reference to the cell object containing the number of times
called. This could have also been written a different way using the
manual, more explicit way, this exercise is left to the reader.

@node Operations on actormaps
@section Operations on actormaps

Using the persistence support on an actormap allows for taking a
portrait of its current state, rehydrating that state into an actormap
and updating the behavior of an object through an upgrade mechanism.

@deffn {Procedure} actormap-take-portrait am persistence-env . roots

Takes a portrait of the objects within the actormap starting with the
supplied roots.

This takes an actormap, @var{am} which contains the objects and an
@var{persistence-env} which describes some additional information about
all the objects within the graph that's needed to support persistence.
The final arguments are all the roots within the graph.

The portrait is a map where each object within the portrait that is
represented by an integer which maps to its respective portrait data (in
a serializable form), this integer is known as a slot. When an object
contains a reference to another object, that object is referred
to by its slot.

The procedure returns two values to its continuation, those are:

1. the self portrait data
2. the root slots: these are in order the integers of the root objects.

@end deffn

@deffn {Procedure} actormap-restore! am persistence-env portrait-data roots

This restores the objects in the actormap @var{am} from the portrait data
@var{portrait-data} using the object specifications data stored in
@var{persistence-env} with the given @var{roots}.

Due to the way rehydration of depictions works each object is
represented as a vow which will be resolved when returned.

This procedure returns to the continuation all the references to the
restored objects for the given @var{roots}. This is so that the number
of values returned is equal to the number of roots provided.

@end deffn

@deffn {Procedure} actormap-replace-behavior! am persistence-env

Replace actors in actormap @var{am} with new behavior from
@var{persistence-env}

It's often desirable, especially when developing a system, to update or
otherwise change the behavior of objects which already exist in an
actormap. This can easily be done by using the persistence system in
Goblins. Each object can describe itself through its self portrait and
then be rehydrated into its new behavior allowing seamless upgrades.

This procedure will first calculate all of the objects that have an
updated constructor within @var{persistence-env}. It will then take a
portrait all those objects, then using that portrait data, rehydrate
them with the new behavior keeping the object references the same.

It's important to note that in order to be able to replace the
behavior the objects must be been defined as redefinable, this is
commonly done by using the @code{define-hackable} macro provided with
Goblins. Non-redefinable actors will not have their behavior replaced
as they will appear as different objects within the
@var{persistence-env}.

@end deffn

@deffn {Procedure} actormap-replace-behavior am persistence-env

Functional version of @code{actormap-replace-behavior!}

This works the same as `actormap-replace-behavior!' but it returns a
transactormap which the user can choose whether or not to commit.

@end deffn

@deffn {Procedure} actormap-save-to-store! am persistence-env store . roots

Take and save the portrait for the object graph starting with @var{roots}
as the root objects in the graph. Store the portrait data within the
@var{store}, which is a @xref{Persistence stores}.

The @var{persistence-env} should be a persistence environment with the object
specifications of all the objects within the object graph being persisted.

@end deffn

@deffn {Procedure} actormap-restore-from-store! am persistence-env store

Restores the object graph that has previously been stored in the @var{store}
into the actormap @var{am}. All objects that appear in the graph must be defined
in the persistence environment @var{persistence-env}. Returns the roots as
values.

@end deffn

@deffn {Procedure} local-refr->persistable-object-identifier local-refr

Convert a local object reference, @var{local-refr} to a persistable object
identifier.

A persistable object identifier is a value which can be used when comparing
two objects together with @code{equal?}. This is particularly useful when using
object refrs as keys within hash tables. They are persistable by Goblins'
persistence system and available immediately upon restoration, without requiring
any coordination with other objects or vats, as is the case when persisting
references to objects on other vats.

This identifier is not a refr and cannot be used to send messages to the object.
It is solely designed as a stand-in to compare two object refrs together to check
if they are the same object or not. Persistable object identifiers also cannot
be converted back to refrs.

This procedure will always convert a local-refr to a persistable object
identifier value which will always be @code{equal?} to another persistable
object identifier produced from the same refr, even between restores.

Note: This only works for refrs spawned in vats, not for refrs from working
directly with actormaps.

@lisp
(use-modules (goblins)
             (goblins core)
             (goblins actor-lib cell)
             (goblins persistence-store memory))

(define store (make-memory-store))
(define-values (vat my-cell)
  (spawn-persistent-vat
   cell-env
   (lambda ()
     (spawn ^cell))
   store))
(define my-cell-identifier
  (local-refr->persistable-object-identifier my-cell))

;; Now make a new actormap, and restore a new cell refr from the portrait data
(define-values (vat* my-cell*)
  (spawn-persistent-vat
   cell-env
   (lambda ()
     (error "should never get here as it's reading from the store"))
   store))
;; Now create an identifier for the my-cell* that we just restored.
(define my-cell-identifier*
  (local-refr->persistable-object-identifier my-cell*))

;;; they're `equal?': (equal? #t)
(pk 'equal? (equal? my-cell-identifier my-cell-identifier*))
@end lisp
@end deffn

@node Persistent vats
@section Persistent vats

When a vat is spawned, it can be spawned as a persistent vat with a
given persistence environment and set of roots. The vat also has a
storage provider associated with it. Vats can be persisted either on
each churn (when the vat reaches quiescence), or manually. If a vat is
configured to persist on churns, just the objects which have changed
(using become) will be persisted.

@deffn {Procedure} spawn-persistent-vat persistent-env spawn-roots-thunk @
       store [#:persist-on] [#:vat-constructor] [#:name] [#:log?]
       [#:log-capacity] [#:persistence-registry] [#:version]
       [#:upgrade]

Create and return a reference to a new @emph{vat} configured with
persistence.

@itemize
@item
@var{persistence-env}: The persistence environment containing all
objects within the persistence graph (@xref{Persistence environments})

@item
@var{spawn-roots-thunk}: A thunk which is invoked if no persisted
state of the vat exists. The lambda should spawn and return the roots
of the object graph to be persisted.

@item
@var{store}: The store that persistence data will be read and written
to @xref{Persistence stores}.

@item
@var{persist-on}: If @code{'churn} will persist on churns, otherwise
in a manual persistence mode (default: @code{'churn}).

@item
@var{vat-constructor}: The constructor used to create the vat,
defaults to @code{spawn-fibrous-vat}.

@item
@var{name}: Name of the vat.  Note that this is @emph{not} the name it is
bound to in the current namespace.

@item
@var{log?}: If @code{#t}, log @emph{vat} events; otherwise, do not.

@item
@var{log-capacity}: Number of events to be retained in the log.
Defaults to 256.

@item
@var{persistence-registry}: The refr to a shared ^persistence-registry
object which, if provided, enables persisting and rehydrating objects
from other local vats.

@item
@var{version}: Usually a number (but not limited to), to indicate the version
for the roots of the graph. When the roots of the graph change, this version
should change and an upgrade path in the @var{upgrade} procedure should be
provided. This defaults to @code{0} when no version is specified.

@item
@var{upgrade}: This is a procedure which is run if the version of the
roots in the persistence store does not match the version in
@var{version}. The procedure takes in the previous version as its
first arguments followed by all the roots that were read from the
store. The procedure should return two values, 1) the new version number
and 2) a list of the new roots themselves. The new root version
should match that of  @var{version}.

The @code{migrations} macro is generally what would be expected to be
used for the upgrade procedure

@end itemize

@lisp
(define-actor (^counter bcom value)
  (lambda ()
    (bcom (^counter bcom (+ 1 value)) value)))

(define persistence-env
  (make-persistence-env
   `((((counter) ^counter) ,^counter))))

(define-values (vat my-counter)
  (spawn-persistent-vat
   persistence-env
   (lambda ()
     (spawn ^counter 0))
   (make-memory-store)))
@end lisp

The @code{^persistence-registry} actor is designed to be spawned anew
each time the program starts (i.e. is not part of the persistence
graph) and is shared amongst several persistent vats which want to
persist/rehydrate objects across vats.

Unlike local near refrs which are promises during rehydration, but are swapped
to local refrs after rehydration is complete, Far refrs remain promises and
should resolve after both vats are setup and have registered themselves with
the @code{^persistence-registry} object.

Here's an example of two vats, the cell on @code{b-vat} holds a reference to
an object on @code{a-vat}

@lisp
;; Spawn the persistence registry in its own vat without persistence enabled.
(define persistence-registry
  (with-vat (spawn-vat)
    (spawn ^persistence-registry)))

;; Spawn the initial a-vat with an a-cell object
(define a-vat-store (make-memory-store))
(define-values (a-vat a-cell)
  (spawn-persistent-vat
   cell-env
   (lambda () (spawn ^cell))
   a-vat-store
   #:persistence-registry persistence-registry))

;; Spawn a b-vat with a b-cell object, that will reference the far refr a-cell.
(define b-vat-store (make-memory-store))
(define-values (b-vat b-cell)
  (spawn-persistent-vat
   cell-env
   (lambda () (spawn ^cell a-cell))
   b-vat-store
   #:persistence-registry persistence-registry))

;; Let go of our persistence registry reference so it can be garbage collected.
(set! persistence-registry #f)
@end lisp

The persistence registry builds a map of strong references to all the
references within a vat. This ensures references only held by far vats
are kept around until all the vats are spawned. A consequence of this
design is that if the reference to the persistence registry is kept
around, the initial graph of actors in vats can never be garbage
collected. Thus, it's important to let go of the reference to the
@code{^persistence-registry} actor once all the vats which need it have
been spawned.

@end deffn

@node Persistence environments
@section Persistence environments

All persistence aware objects need be defined in what's known as an
persistence environment. These environments specify a set of information
which must exist to be able to take portrait and/or rehydrate objects.
This is a unique name for the object, the constructor, and if different
from the constructor, a procedure to rehydrate the object.

When taking portraits of objects or rehydrating them every single object
must be present in the given persistence env, this would obviously be
quite tedious as object graphs get larger and contain lots of different
objects. This is made easier by the persistence environments being able
to extend from (or inherit from) other persistence environments. Lets
take the idea of our greeter from earlier:

@lisp
(use-modules (goblins)
             (goblins actor-lib cell))

(define-actor (^greeter* _bcom our-name number-of-times)
  (lambda (their-name)
    (let ((times ($ number-of-times)))
      ($ number-of-times (+ 1 times))
      (format #f "Hello ~a, my name is ~a (called ~a)"
              their-name our-name times))))

(define* (^greeter bcom our-name #:optional [number-of-times 0])
  (spawn ^greeter* our-name (spawn ^cell number-of-times)))
@end lisp

The persistence environment here would need to specify two objects, the
@code{^greeter*} object and the @code{^cell} object. This is made a
little more complicated by the fact that we didn't define @code{^cell},
we're using that from a library, Goblins' actor-lib library.

This is where persistence environment inheritance helps, we can specify
the objects we've defined (i.e. @code{^greeter*}) and specify the
persistence environment that actor-lib's cell module provides for us:

@lisp
(define greeter-env
  (persistence-env-compose
   (namespace-env (greeter)
     ^greeter*)

   ;; Specify the cell environment from (goblins actor-lib cell)
   cell-env))
@end lisp

If we later wanted to define another persistence environment which uses
our greeter, we could define a new persistence environment which
extends from @var{greeter-env}.

The reason we're including @code{^greeter*}, rather than
@code{^greeter} in the persistence environment, is that when a
constructor returns a refr, rather than a procedure. Goblins throws
away the outer object and just uses the returned refr. As
@code{greeter} is acting as a helper and returning a refr to
@code{greeter*}, it is that object which you're getting when you spawn
@code{^greeter} and thus tha  object you need to put in the
persistence environment.

@deffn {Procedure} make-persistence-env bindings #:extends

This creates a persistence environment based on a set of object
specifications and a list of other persistence environments to extend
from.

This takes two arguments, each being a list. The first is @var{bindings}
which is a list of all the object specifications specified as a list:

(OBJECT-NAME OBJECT-CONSTRUCTOR [OBJECT_REHYDRATOR])

The second keyword argument, @var{extends} is either a single persistent
environment to extend from, or a list of several persistent environments
to extent from.

Important: Care should be taken to prevent cycles within persistence
environments.

This procedure returns a single value, a persistence environment.

@end deffn

@deffn {Procedure} persistence-env-compose envs @dots{}

Composes a new persistence environment which contains all of the
provided @var{envs}.

@end deffn

@deffn {Syntax} namespace-env namespace objects @dots{}

Makes a new persistence environment with all of the objects provided in
@var{objects} namespaced within @var{namespace}.

The following are equivalent:
@lisp
(define cell-env
  (make-persistence-env
   `((((goblins actor-lib cell) ^cell) ,^cell)
     (((goblins actor-lib cell) ^ro-cell) ,^ro-cell)
     (((goblins actor-lib cell) ^wo-cell) ,^wo-cell))))

(define cell-env
  (namespace-env
   (goblins actor-lib cell)
   ^cell ^ro-cell ^wo-cell))
@end lisp
@end deffn

@node Upgrading through persistence
@section Upgrading through persistence

The design of the persistence system allows for this to be a powerful
mechanism to make upgrades for object data. The very basic situation of
adding a new piece of data is automatic as you can provide a default
value. More complicated data upgrade is also fully supported. Lets take
an example of a thermostat that represents the set temperature in
fahrenheit:

@lisp
(define (^thermostat bcom temperature)
  (define main-beh
    (methods
     [(get-current-temp) temperature]
     [(set-temp new-temp) ...]))
  (define (self-portrait)
    (list temperature))
  (portraitize main-beh self-portrait))

(define thermostat-env
  (make-persistence-env
   (list (list '((house temp) ^thermostat) ^thermostat))))
@end lisp

Later, we want to change the thermostat object to work with the set
temperature in Celsius. Normally this is a bit tricky as we have just
stored an integer for the portrait data, however Goblins tags this data
with a version and it can be upgraded and tagged by us too, lets take a
look at how we can upgrade the above actor using this upgrade mechanism:

@lisp
(define (^thermostat bcom temperature)
  (define main-beh
    (methods
     [(get-current-temp) temperature]
     [(set-temp new-temp) ...]))
  (define (self-portrait)
    (versioned 1 (list temperature)))
  (portraitize main-beh self-portrait))

(define (rehydrate-thermostat version restored-temperature)
  (define temperature
    (match version
      ;; F -> C conversion
      [0 (+ (* 9/5 restored-temperature) 32)]
      ;; C, no conversion needed.
      [1 restored-temperature]))
  (spawn ^thermostat temperature))

(define thermostat-env
  (make-persistence-env
   (list (list '((house temp) ^thermostat) ^thermostat rehydrate-thermostat))))
@end lisp

There are two changes here:

1. The thermostat object's @code{self-portrait} function now versions
the data with version 1.
2. We now have this @code{rehydrate-thermostat} function which takes in
a version and might convert it before spawning the thermostat.

Goblins has a @code{versioned} function provided which allows the first
argument to be a version tag which usually is an integer but could be
any atomic serializable data such as a symbol or even string and as a
second argument the self-portrait data. If the self-portrait doesn't
return a versioned portrait, Goblins will assign the version 0 to that
portrait data. Later you're able to match on it and upgrade it like the
example above.

With the above example newly created objects which are created using
Celsius will not be need to be upgraded, but the old fahrenheit objects
will be converted.

@node Persistence stores
@section Persistence stores

Persistence stores are an abstraction used primarily for vats, however
they can also be used with actormaps too. They are used to save and
retrieve data from different backing stores and other storage
mechanisms.

Currently Goblins provides two stores, the first is an in memory store
where nothing is persisted and the second is a syrup store, which
reads and writes to files, serializing the data in the Syrup
serialization format.

Note: Stores are currently not designed in a thread safe manner so
care should be taken to ensure they are only accessed by one vat.

The procedure @code{persistence-store-copy!} from @code{(goblins)}
can be used to convert between any two stores. This can be done with:

@lisp
(use-modules (goblins)
             (goblins persistence-store syrup)
             (goblins persistence-store bloblin))

(define old-store (make-syrup-store "my-old-store.syrup"))
(define new-store (make-bloblin-store "new-bloblin-store"))
(persistence-store-copy! old-store new-store)
@end lisp

@node Bloblin Store
@subsection Bloblin Store

This provides a fast and efficient Syrup based store. Goblins vats are usually
efficient providing only the deltas (i.e. the portraits of changed objects),
rather than the full graph. This store is able to be fast by streaming these
deltas to the file. This is much faster than writing out the full graph each
time an object changes (or rather for each churn).

The store is provided by @code{(goblins persistence-store bloblin)}.

@deffn {Procedure} make-bloblin-store bloblin-dir [#:deltas-per-file] [#:max-bloblin-files]

Return a memory store which will write its data to @var{bloblin-dir}. This
directory is expected to be used solely for Bloblin's data to exist within. The
number of deltas streamed to a single file before a new file is written to can
be configured with @var{deltas-per-file}, by default it occurs every 1000
deltas. If @var{deltas-per-file} is set to @code{#f}, then creating a new file
based solely on the deltas will be disabled. During normal operation Goblins
tries to be efficient using only deltas, but Goblins sometimes does still need
to save a full graph (e.g. when upgrading the roots of a graph). When Goblins
saves a full graph, Bloblin will always create a new file regardless of what
@var{deltas-per-file} is set to.

If @var{max-bloblin-files} is specified, only @var{max-bloblin-files} number of
bloblin files will be kept (deleting the older bloblin files). By default this
is @code{#f}, which disables deletion of older bloblin files allowing them to
accumulate over time.

@end deffn

@node Memory store
@subsection Memory store

The memory store keeps everything within memory and does not persist
it. This is mostly good for testing.

The memory store can be found in @code{(goblins persistence-store
memory)}.

@deffn {Procedure} make-memory-store

Returns a memory store.

@end deffn

@node Syrup store
@subsection Syrup store

This provides a persistent storage mechanism which serializes the data
to a file in syrup format.

The syrup store can be found in @code{(goblins persistence-store
syrup)}.

More information about the Syrup serialization format can be found
here: @uref{https://github.com/ocapn/syrup}

@deffn {Procedure} make-syrup-store backing-filename

Returns a syrup store which stores the data in a file specified in
@var{backing-file}.

@end deffn


@node Web localStorage store
@subsection Web localStorage store

The @code{(goblins persistence-store local-storage)} module provides a
persistence store which uses the
@url{https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage,Web localStorage interface}.
This store is thus only supported on web targets using Hoot and not the Guile VM.

@deffn {Procedure} make-local-storage-store name

Return a local storage store which stores the data in the web browser's local
storage, under the @var{name} key (must be a string).

@end deffn

@node Sleepy actors
@subsection Sleepy actors

Goblins has the ability to have actors enter a sleep state, and wake
back up when they're needed. This allows the persistence system to act
somewhat like a database's working set or a hot cache, where actors do
not need to take up resources when not in use. When an actor is asleep
only the reference and a small subset of the object remains. The
local references an actor holds within its portrait are kept around in
memory, however all the other portrait data as well as the behaviour
closure will be freed. This allows sleeping actors to have a much
smaller footprint.

This functionality is particularly useful if, for example, you were to
have a blog where all posts are actors. Much of the data is inert
(largely textual in the case of blogs) and can be kept on disk, woken up
only when needed. This would also be good for chat logs ActivityPub
implementations, and many other situations.

Goblins code can be written as normal using @code{$}, @code{<-}, or
@code{<-np} with references (asleep or awake). If an object is asleep
then it'll be restored to handle the message. After each churn, the vat
will query the sleep strategy to determine which (if any) objects should
be put to sleep. Goblins comes with sleep strategies out of the box. The
sleep strategy is determined per vat when spawning.

To use sleepy actors with a vat, you can enable it by simply specifying
a sleep strategy. Here's how to use the @code{sleep:always}
included with Goblins:
@lisp
(use-modules (goblins)
             (goblins persistence-store memory)
             (goblins sleep-strategy always)
             (goblins actor-lib cell))

(define store (make-memory-store))
(define-values (vat cell)
  (spawn-persistent-vat
   cell-env
   (lambda () (spawn ^cell (spawn ^cell 'inner)))
   store
   #:sleep-strategy sleep:lru))
@end lisp

When a brand new vat is spawned, it initially spawns the objects from the
thunk provided. These will initially be awake, after the first churn
the sleep strategy will determine which actors will be put to sleep. When
restoring a vat, all the actors will be restored asleep. Actors will then be
woken up only after messaging them.


@node Sleep strategies
@section Sleep strategies

Sleep strategies are what determine which objects should be kept awake
and which should be put to sleep at the end of each churn. They are
informed by Goblins when and how an object is used and then asked to
provide a list of the objects which should be put to sleep. This
allows for many different strategies to be used, depending on the
needs of the program. It also allows bespoke strategies to be defined
and used where existing ones aren't sufficient.

@node Least recently used (LRU)
@subsection Least recently used (LRU)

The least recently used strategy (provided by the @code{(goblins
sleep-strategy lru)} module) maintains a fixed size cache of awake
objects.  When the cache is full, the least recently messaged objects
are expired from the cache and then put to sleep.  LRU is a good
one-size-fits-most strategy; specialized workloads may require a
different strategy but LRU is a safe default choice.

@deffn {Procedure} sleep:lru [#:max-objects]
Return a new instance of the LRU sleep strategy that retains
@var{max-objects} in its internal cache.  When the cache is full, the
least recently used objects will be put to sleep.
@end deffn

@node Always
@subsection Always

The ``always'' strategy (provided by the @code{(goblins sleep-strategy
always)} module) will put all awake objects to sleep at the earliest
opportunity.  This strategy exists for testing purposes and shouldn't
be used in production.

@deffn {Procedure} sleep:always
Return a new instance of the ``always'' sleep strategy.
@end deffn
