path: root/tests/mountpoints.txt

Mountpoints are very much like traditional *NIX filesystem mounts. The
intention is to allow more than one backingstore (stable storage
device) to become part of a datastore at runtime. This is done by
mounting a backingstore on the datastore.

(clean up)
>>> import os
>>> assert os.system('rm -rf /tmp/store1/') == 0
>>> assert os.system('rm -rf /tmp/store2/') == 0
>>> assert os.system('rm -rf /tmp/store3/') == 0


>>> from olpc.datastore import DataStore
>>> from olpc.datastore import backingstore
>>> from testutils import tmpData
>>> import dbus


Here we create a datastore, and mount a backingstore on tmp. By
default this will create a new directory in /tmp which will then be
used for storage.

>>> ds = DataStore()
>>> ds.registerBackend(backingstore.FileBackingStore)
>>> mp1 = ds.mount("/tmp/store1", dict(title="Primary Storage"))

This will list all the mount points. It returns a list of dicts with
the minumum keyset of 'id', 'uri', and 'title'. Title is the Human
readable name of the mount. 'Id' is the most important property this
can be used to control the storage target or to filter results.

>>> mps = ds.mounts()
>>> mountpoint = mps[0]['id']


Now lets create some content

>>> u1 = ds.create(dict(title="Document 1", filename="one.txt"), tmpData("""document one"""))
>>> u2 = ds.create(dict(title="Document 2", mime_type="text/plain"), tmpData("""document two"""))

We can now, if we wish verify which mount point this content came
from.

>>> ds.complete_indexing()

>>> c1 = ds.get(u1)
>>> assert c1.backingstore.id == mountpoint

However this interface isn't available over DBus and objects are
normally located and inspected using the find() method.

>>> c1a = ds.find(dict(title="Document 1"))[0][0]
>>> assert c1a['mountpoint'] == mountpoint

We can see that the mountpoint property was mapped on the object and
refers to the proper storage.

Now lets add another mount point.

>>> mp2 = ds.mount("/tmp/store2", dict(title="Secondary Storage"))

Now lets create a new content item.
>>> u3 = ds.create(dict(title="Document 3", mountpoint=mp2), tmpData("""document three"""))

>>> ds.complete_indexing()

We explictly passed a mount point here. Lets examine the properties of
the object and verify this.
>>> c3 = ds.find(dict(title="Document 3"))[0][0]
>>> assert c3['mountpoint'] == mp2

Now lets filter a find call to only selected mountpoints.

>>> results, count = ds.find(dict(mountpoints=[mp1]))
>>> assert count == 2

>>> results, count = ds.find(dict(mountpoints=[mp2]))
>>> assert count == 1

>>> results, count = ds.find({})
>>> assert count == 3

We can see that filtering by mount point works as expected.

Now we are going to create an inplace mount. This is designed around
USB keys and the like. In this case we want to leave files in place,
there will be no working copies.

First lets create some inplace data that we expect to get imported and
indexed.

>>> os.makedirs("/tmp/store3/nested")
>>> fp = open("/tmp/store3/doc4.txt", "w")
>>> fp.write("This is document four")
>>> fp.close()

>>> fp = open("/tmp/store3/nested/doc5.txt", "w")
>>> fp.write("This is document five")
>>> fp.close()

Register the filesystem type
>>> ds.registerBackend(backingstore.InplaceFileBackingStore)

>>> mp3 = ds.mount("inplace:/tmp/store3", dict(title="Fake USB"))

If that worked it should have imported content on load().

>>> ds.complete_indexing()

>>> result, count = ds.find(dict(fulltext="four"))
>>> assert count == 1
>>> assert result[0]['mountpoint'] == mp3


Let's unmount 'Fake USB' and then remount it with some options passed
as DBus data.

>>> ds.unmount(mp3)

>>> mp3 = ds.mount("inplace:/tmp/store3", dict(title=dbus.String("Fake USB again"),  
...                                            sync_mount=True))

>>> ds.complete_indexing()


>>> result, count = ds.find(dict(fulltext="four"))
>>> assert count == 1
>>> assert result[0]['mountpoint'] == mp3

>>> mp = ds.mountpoints[mp3]

Check for the new value in the descriptor

>>> assert mp.descriptor()['title'] == 'Fake USB again'

Verify that we can get the properties of objects on the inplace
stores.

>>> uid = result[0]['uid']
>>> props = ds.get_properties(uid)
>>> assert props['title'] == "doc4"


Currently sugar defines doing a copy as zeroing out the uid and
changing the mountpoint. Lets copy an object from mp3 to mp1, the
primary store.
>>> props['mountpoint'] = mountpoint 
>>> fn = ds.get_filename(uid)

>>> copyuid = ds.create(props, fn)

>>> ds.complete_indexing()


>>> result, count = ds.find(dict(fulltext="four"))
>>> assert count == 2

We also need to test that we can copy from a normal store to an
inplace one. Lets move the object with u1 to mp3

>>> props = ds.get_properties(u1)
>>> props['mountpoint'] = mp3
>>> pen_copy = ds.create(props, ds.get_filename(u1))

>>> ds.complete_indexing()

>>> result, count = ds.find(dict(mountpoints=[mp3], filename="one.txt"))
>>> assert count == 1
>>> assert result[0]['uid'] == pen_copy

The file was properly created in the expected place.

>>> assert os.path.exists('/tmp/store3/one.txt')


We also need to be sure that delete commands work on inplace
mounts. We will delete the object from the datastore and then verify
that the file is missing.

>>> ds.delete(pen_copy)
>>> ds.complete_indexing()

>>> os.path.exists('/tmp/store3/one.txt')
False


Now a tricky case where we corrupt the metadata on a mount and want to
verify that we can still remount the store.

>>> ds.unmount(mp3)
>>> fp = open('/tmp/store3/.olpc.store/metainfo', 'w')
>>> fp.seek(0)
>>> fp.write('broken')
>>> fp.close()

>>> mp3 = ds.mount("inplace:/tmp/store3", dict(title="Fake USB from broken"))
>>> mp = ds.mountpoints[mp3]
>>> assert mp.descriptor()['title'] == 'Fake USB from broken'


>>> ds.stop(); del ds