WWR - Railsplugin: Acts As Replica

Acts As Replica

ActsAsReplica

This plugin is meant to be used in offline-client scenarios where the same Rails app is deployed in both the clients and in the main server. For instance using some other 3rd party solution as Joyent's Slingshot, Rails2Ext and so forth.

Bear in mind that this is not a one-package-solves-all kind of solutions. It assumes the scenario of multiple offline clients and one master server. It doesn't replace heavy industrial level message queues or database level merge replication. It also doesn't support master-less distributed peer-to-peer replications. Only N-clients-1-master is supported by now.

Clients can input data offline. This data will be recorded in a local sqlite3 file. Then it can connect to the server to pull more recent data from it and push its new data back to it.

Dependencies

- ./script/plugin install engines - ./script/plugin install userstamp - gem install uuidtools

Project Assumptions

This plugin follows several assumptions:

- Every replicable table has to have a Surrogate UUID-based primary key This is made this way to avoid any possible primary key conflict between the clients or server. Yes, I could use integer ranges for each client but this would add unnecessary overhead to the process. I could also have made some man-in-the-middle controller that would transact ids back and forth, but this would be even more unnecessary overhead.

- This app has to have a User class with a singleton 'current_user' method. The app has to make sure User.current_user always contain something (usually with the before_filter method in the controller to get the currently logged in user)

- The primary key of the User model also has to be a UUID, and it also has to have a secondary UUID (column named GUID) that has to be available at the RemoteClient model in the server. It means that the server doesn't need to have a full User table with all the offline clients if it doesn't want to (this may make the deployment process easier). And finally, this User model also has to have a last_synced timestamp column to record the syncing dates.

- Every replicable table has to have UserStamps (created_by, created_at, updated_by, updated_at) because this plugin uses this data to know how to track them. So, it's not optional. The detail being the the created_by and updated_by columns will hold the UUID primary key of the User.

- The client can be behind a http proxy, using SSL connection and the web server can request basic authentication credentials. Configurations can be held in the config/syncable.yml file. Be careful though, as it supports the same infra-structure as Net::HTTP, so probably Windows based servers need more tests as they are usually not standards compliant.

- It doesn't use XML for the payload packages for 2 reasons: first of all, I don't personally like XML for data transfer. Second of all, YAML is lighter weight, supported through all Ruby and Rails objects nativelly and easily human readable. One can make an adapter later, as this is only a matter of marshalling. So it may not be very easy to place message brokers in between the client and server. But as I said, this is a very opinionated piece of software made for my own use.

Basic Workflow (started through /syncs/perform_sync in the client)

(1) The client initiate a handshake process:

GET /syncs/handshake.yaml

(2) The server creates an internal session and sends back both a cookie ID (session ID) and a challenge key.

(3) The client has to look for its internal users’s GUID and create a response to the challenge:

POST /syncs/handshake.yaml?client_id= (4) The server has the user’s GUID mapped in the RemoteClient table so it can compare the received response with its own. When the server receives new data from the client, it records this timestamp in the RemoteClient table, for each client. So, it sends this date back.

(5) Now, the client requests the most recent data from the server. It has to look for the last_synced column in its own User table.

POST /syncs/down.yaml&for_when=9999-99-99T99:99:99-99:99

(6) Server calls Sync.down internally and looks for all new data since the ‘for_when’ timestamp received that was not created by the logged in user. Sends back a ActsAsReplica::Structs::SyncPayload package encoded as YAML.

(7) Client calls Sync.up internally to record the new data. If everything goes fine, records the newest last_synced timestamp in the User table.

(8) Client calls Sync.down internally, using the date obtained from the server upon the handshake described above. It retrieves the newest data it has created offline and also creates a ActsAsReplica::Structs::SyncPayload package that it posts to the server in YAML format:

POST /syncs/up.yaml?syncs=<yaml::object>

(8) Server calls Sync.up internally and processes the received package. If everything goes fine, it updates the last_synced column in the RemoteClient table for this particular logged in user.

(9) Client compiles the results page with all that happened in this transaction

== INITIAL TESTS

As this involves at least two peers, we have to load up at least two mongrel processes. In this particular test, we’ll use the development and production environments at once as a testbed for a simple scenario.

(1) First, everytime we want to test the whole scenario, we have to clean the databases. Migrations are already set to correctly populate both different environments. So, from the shell:

rm db/.sql; rake db:migrate RAILS_ENV=development; rake db:migrate RAILS_ENV=production

(2) Now, we start 2 mongrel processes in 2 different shells:

./script/server -p 3000 -e development ./script/server -p 3001 -e production

(3) Now, login with username ‘admin’, password ‘admin’ at:

http://localhost:3000/users/login

(4) Then manually type this URL:

http://localhost:3000/syncs/perform_sync

(5) The call above simulates a client starting synchronization with a server. If everything went fine, we can get in the ./script/console [environment] of each and check that totals for ReturnOrder.count and Batch.count are the same in both environments. The browser should disclose something similar to this:

Perform Syncing Results:

Config:

:handshake_uri: /syncs/handshake.yaml :port: 3001 :hostname: localhost :down_uri: /syncs/down.yaml :protocol: http :client_version: 0.0.1 :up_uri: /syncs/up.yaml

Result Up:

- !ruby/object:ActsAsReplica::Structs::SyncUpResult errors: []

last_updated_at: 2007-07-04 17:22:48.733361 -03:00 status: 200 total: 0

Result Down:

- !ruby/object:ActsAsReplica::Structs::SyncUpResult errors: []

last_updated_at: 2007-07-04 17:22:48.735303 -03:00 status: 200 total: 0