Page tree
Skip to end of metadata
Go to start of metadata

ResourceOrigin#watchForChanges API allows to subscribe to the changes in the resource hierarchy => in order to fulfil it for the default set of Magnolia resource origins (classpath, JCR, FS) - each of them has to be supplied with a monitoring interface of the respective underlying datasource.

In case of JCR the change monitoring and communication can be handled fairly reliably via ObservationUtil, classpath resource origin relies on a custom ClasspathService and => under our full control. File system resource origin is a more special case since Java capabilities of directory observation (so called WatchService, see this tutorial) turn out to be dependent on the target platform.

Here we provide an aggregated matrix which describes the event sequences generated by WatchService in response to various common file operations on Win7, Ubuntu14 and OS 10.10 Yosemite:

 Events reported (in terms of java.nio.file.StandardWatchEventKinds, oddities are marked with (warning))
Actions

Windows 7

Windows 10

Linux (Ubuntu 14)

OS X Yosemite / El Capitan / Sierra

Linux (Fedora 24)

mkdir foo

// Create folder

  • ENTRY_CREATE<foo>

dittodittodittoditto

mv foo bar

// Rename folder

  • ENTRY_DELETE<foo>

  • ENTRY_CREATE<bar>

dittodittodittoditto

rm -rf foo

// Delete folder

  • ENTRY_DELETE<foo>

dittodittodittoditto

touch foo/bar

// Create file

  • ENTRY_CREATE<bar>

  • ENTRY_MODIFY<foo>

  • ENTRY_CREATE<bar>

  • ENTRY_MODIFY<foo>

  • (warning) ENTRY_CREATE<bar>

  • ENTRY_CREATE<bar>

  • ENTRY_MODIFY<foo>

  • ENTRY_CREATE<bar>

  • ENTRY_MODIFY<foo>

rm foo/bar

// Delete file

  • ENTRY_DELETE<bar>

  • ENTRY_MODIFY<foo>

  • (warning)ENTRY_DELETE<bar>

  • (warning) ENTRY_DELETE<bar>

  • ENTRY_DELETE<bar>

  • ENTRY_MODIFY<foo>

  • (warning) ENTRY_DELETE<bar>

mv foo/bar foo/baz

// Rename file

  • (warning)

  • ENTRY_REMOVE<bar>

  • ENTRY_CREATE<baz>

  • ENTRY_MODIFY<baz>

  • ENTRY_MODIFY<foo>

  • (warning)

  • ENTRY_REMOVE<bar>

  • ENTRY_CREATE<baz>

  • (warning)

  • ENTRY_REMOVE<bar>

  • ENTRY_CREATE<baz>

  • (warning)

  • ENTRY_REMOVE<bar>

  • ENTRY_CREATE<baz>

  • ENTRY_MODIFY<foo>

  • (warning)

  • ENTRY_REMOVE<bar>

  • ENTRY_CREATE<baz>

cp foo/baz bar/baz

// Copy file to another folder

  • (warning)

  • ENTRY_CREATE<bar/baz>

  • ENTRY_MODIFY<bar/baz>

  • ENTRY_MODIFY<bar>

  • (warning)

  • ENTRY_CREATE<bar/baz>

  • ENTRY_MODIFY<bar/baz>

  • ENTRY_MODIFY<bar>

  • ENTRY_CREATE<bar/baz>

  • ENTRY_MODIFY<bar/baz>

  • ENTRY_CREATE<bar/baz>

  • ENTRY_MODIFY<bar/baz>

    • ENTRY_CREATE<bar/baz>

    • ENTRY_MODIFY<bar/baz>

mv bar/foo baz/foo

// Move file to another folder

  • (warning)

  • ENTRY_DELETE<bar/foo>

  • ENTRY_CREATE<baz/foo>

  • ENTRY_MODIFY<baz/foo>
  • ENTRY_MODIFY<bar>
  • ENTRY_MODIFY<baz>
  • (warning)

  • ENTRY_DELETE<bar/foo>

  • ENTRY_CREATE<baz/foo>

  • ENTRY_MODIFY<baz>
  • (warning)

  • ENTRY_DELETE<bar/foo>

  • ENTRY_CREATE<baz/foo>

  • (warning)

  • ENTRY_DELETE<bar/foo>

  • ENTRY_CREATE<baz/foo>

  • ENTRY_MODIFY<bar>
  • ENTRY_MODIFY<baz>

  • (warning)

  • ENTRY_DELETE<bar/foo>

  • ENTRY_CREATE<baz/foo>

mv bar/foo baz/foo

// Move non-empty sub-folder

  • ENTRY_DELETE<bar/foo>

  • ENTRY_CREATE<baz/foo>

  • ENTRY_MODIFY<baz>

  • ENTRY_DELETE<bar/foo>

  • ENTRY_CREATE<baz/foo>

  • ENTRY_MODIFY<baz>
  • (warning)

  • ENTRY_DELETE<bar/foo>

  • ENTRY_CREATE<baz/foo>

  • ENTRY_DELETE<bar/foo>

  • ENTRY_CREATE<baz/foo>

  • ENTRY_MODIFY<baz>
  • (warning)

  • ENTRY_DELETE<bar/foo>

  • ENTRY_CREATE<baz/foo>

Takeaways

Which useful conclusions can we make so far?

  • None of the three platform implementations is completely identical to the others. However, the differences are not dramatical.
  • Since OS X WatchService implementation uses polling instead of the direct consumption of native events - there's significant latency of the occurrence of events in Java app .
  • Linux platform seems to be most moderate in terms of the amount of events it generates (which is good).
    • Windows/OS X generate seemingly redundant folder modification events in response to folder content modifications (file/sub-directory deletions/creations).
    • Windows version also produces modification event along with creation event (seems totally redundant).
    • Can/should we try to get rid of such event noise? 
      • Maybe folder modification events should be completely ignored?
      • A small async executor/buffer could help to filter the redundant events (could be implemented as an optimisation action).
  • (warning) When folder is deleted - an event is generated only for the folder entry, all sub-folders may not be accessed. This might be a serious issue for e.g. configuration framework - origin would no longer be able to resolve the contents of a removed folder and configuration source wouldn't be able to determine which definitions should be unregistered.
    • Same happens for newly created folders, but situation is simpler to handle since folder files and subdirectories exist and can be traversed either within FileWatcherCallback or in a callback which processes the event (see MAGNOLIA-6278).
    • Possible way out - make configuration source track the definitions it registers.
    • Question - does recursive folder content traversal ever makes sense? I.e. we can go around all the entries which came along with a new folder and generate 'created' events for each entry, but we cannot do the same for the case of a deleted folder => maybe we should not do that in any case and rely on the subscribers to handle the situation on their own? 

      • This however, is not entirely trivial when dealing with the LayeredResourceOrigin.
  • No labels