Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calendar synchronisation #125

Merged
merged 31 commits into from
Jul 14, 2016
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
f3f059e
Drop `update_calendar` and associated code.
Jul 8, 2016
6a4a0a4
Rename Workers::Synchronisation to Workers::PropertySynchronisation.
Jul 8, 2016
982a829
Introduce background workers, configurable per supplier.
Jul 11, 2016
4dd4ee3
Ability to filter workers for a given supplier.
Jul 11, 2016
61c05a8
Include flows for creating suppliers and associated workers.
Jul 11, 2016
afb4826
Drop the contradicting on_delete: set null option on foreign keys.
Jul 11, 2016
5438de3
Flexible supplier sync workers configuration via suppliers.yml.
Jul 11, 2016
74dff28
Drops `next_run_at` column from hosts.
Jul 11, 2016
76dd403
Rename supplier creation flow to host creation flow.
Jul 12, 2016
7393342
Ability to query for hosts by identifier and supplier.
Jul 12, 2016
cdbc667
Query for backround workers associated with a host instead of supplier.
Jul 12, 2016
85d8939
Create background workers associated with hosts.
Jul 12, 2016
e046e38
Revert suppliers rake task to simplicity.
Jul 12, 2016
e004485
Run host creation in a database transaction.
Jul 12, 2016
1e66a17
Include rake task to update workers definition for all hosts.
Jul 12, 2016
e04b467
Support querying for pending background workers.
Jul 12, 2016
12c3cae
Schedule workers rather than hosts.
Jul 12, 2016
df7a08b
Process background workers instead of sync operations.
Jul 12, 2016
f985e48
Background worker coordination by the queue processor.
Jul 12, 2016
a3a4294
Update AtLeisure event name according to recent changes.
Jul 12, 2016
5041bf7
Introduce Roomorama::Calendar to specify rates and availabilities.
Jul 12, 2016
45be50a
Add new UpdateCalendar Roomorama API operation.
Jul 12, 2016
9e39461
Include type of synchronisation worker when sync process starts.
Jul 12, 2016
5050890
Add stats and type to sync processes.
Jul 13, 2016
d7831ce
Migrate to stats field on property synchronisation.
Jul 13, 2016
0d1d969
Validate calendars when required.
Jul 13, 2016
d230af3
Operation runner for calendar updates.
Jul 13, 2016
bd1ff84
Introduce a CalendarSynchronisation helper class.
Jul 13, 2016
267b475
Permanently delete property counters from sync_processes.
Jul 13, 2016
9600f7b
Update change log.
Jul 13, 2016
cc681fb
Documentation fixes.
Jul 14, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Include flows for creating suppliers and associated workers.
  • Loading branch information
Renato Mascarenhas committed Jul 12, 2016
commit 61c05a8a1405d2dce7a45276f86d4ab77857e619
16 changes: 16 additions & 0 deletions lib/concierge/entities/background_worker.rb
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,22 @@
class BackgroundWorker
include Hanami::Entity

# possible values for the +type+ column of a background worker:
#
# +metadata+ - where processing of property metadata is fetched, parsed and
# synchronised with Roomorama. Includes property images.
# +availabilities+ - processing of the calendar of availabilities for a property.
# Indicates availabilities and prices.
TYPES = %w(metadata availabilities)

# possibile statuses a worker can be in:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

#
# +idle+ - the background worker is not being run, and the +next_run_at+ column
# stores a timestamp in the future.
# +running+ - the background worker is currently running and therefore should not
# be rescheduled.
STATUSES = %w(idle running)

attributes :id, :supplier_id, :next_run_at, :interval, :type, :status,
:created_at, :updated_at
end
2 changes: 1 addition & 1 deletion lib/concierge/flows/external_error_creation.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module Concierge::Flows

# +UseCases::ExtennalErrorCreation+
# +Concierge::Flows::ExternalErrorCreation+
#
# This use case class wraps the creation of an +ExternalError+ record, performing
# attribute validations prior to triggering the database call.
Expand Down
121 changes: 121 additions & 0 deletions lib/concierge/flows/supplier_creation.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
module Concierge::Flows

# +Concierge::Flows::BackgroundWorkerCreation+
#
# This class encapsulates the flow of creating a background worker for a given
# supplier, validating the parameters and interpreting human-readable interval
# times, such as "1d" for one day, and generating the resulting number of seconds
# to be stored in the database.
class BackgroundWorkerCreation
include Hanami::Validations

INTERVAL_FORMAT = /^\s*(?<amount>\d+)\s*(?<unit>[smhd])\s*$/

attribute :supplier_id, presence: true
attribute :interval, presence: true, format: INTERVAL_FORMAT
attribute :type, presence: true, inclusion: BackgroundWorker::TYPES
attribute :status, presence: true, inclusion: BackgroundWorker::STATUSES

def perform
if valid?
worker = find_existing || build_new

# set the values passed to make sure that changes in the parameters update
# existing records.
worker.interval = interpret_interval(interval)
worker.type = type
worker.status = status

worker = BackgroundWorkerRepository.persist(worker)
Result.new(worker)
else
Result.error(:invalid_parameters)
end
end

private

def find_existing
workers = BackgroundWorkerRepository.for_supplier(supplier).to_a
workers.find { |worker| worker.type == type }
end

def build_new
BackgroundWorker.new(supplier_id: supplier_id)
end

def interpret_interval(interval)
match = INTERVAL_FORMAT.match(interval)
absolute = match[:amount].to_i

case match[:unit]
when "s"
absolute
when "m"
absolute * 60
when "h"
absolute * 60 * 60
when "d"
absolute * 60 * 60 * 24
end
end

def supplier
@supplier ||= SupplierRepository.find(supplier_id)
end

def attributes
to_h
end
end

# +Concierge::Flows::SupplierCreation+
#
# This class encapsulates the creation of supplier, a including associated
# background workers. Suppleirs are composed of a +name+ only. A definition of
# the workers is also expected by this class.
class SupplierCreation
include Hanami::Validations

attribute :name, presence: true
attribute :workers, presence: true

# creates database records for the supplier and background workers.
# Returns a +Result+ instance wrapping the resulting +Supplier+ instance,
# or an error in case the parameters are not valid.
def perform
if valid?
name = attributes[:name]
supplier = SupplierRepository.named(name) || create_supplier(attributes[:name])

attributes[:workers].each do |type, data|
result = BackgroundWorkerCreation.new(
supplier_id: supplier.id,
interval: data[:every],
type: type.to_s,
status: "idle"
).perform

return result unless result.success?
end

Result.new(supplier)
else
Result.error(:invalid_parameters)
end
end

private

def create_supplier(name)
SupplierRepository.create(
Supplier.new(name: attributes[:name])
)
end

def attributes
to_h
end
end

end
140 changes: 140 additions & 0 deletions spec/lib/concierge/flows/supplier_creation_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
require "spec_helper"

RSpec.describe Concierge::Flows::SupplierCreation do
let(:parameters) {
{
name: "Supplier X",
workers: {
metadata: {
every: "1d"
},
availabilities: {
every: "2h"
}
}
}
}

subject { described_class.new(parameters) }

describe "#perform" do
it "returns an unsuccessful result without a valid name" do
[nil, ""].each do |invalid_name|
parameters[:name] = invalid_name

result = subject.perform
expect(result).to be_a Result
expect(result).not_to be_success
expect(result.error.code).to eq :invalid_parameters
end
end

it "returns an unsuccessful result without a workers definition" do
parameters.delete(:workers)

result = subject.perform
expect(result).to be_a Result
expect(result).not_to be_success
expect(result.error.code).to eq :invalid_parameters
end

it "returns an unsuccessful result if parameters for the workers are missing" do
parameters[:workers][:metadata].delete(:every)

result = subject.perform
expect(result).to be_a Result
expect(result).not_to be_success
expect(result.error.code).to eq :invalid_parameters
end

it "returns an unsuccessful result if the worker type is unknown" do
parameters[:workers][:invalid] = parameters[:workers].delete(:metadata)

result = subject.perform
expect(result).to be_a Result
expect(result).not_to be_success
expect(result.error.code).to eq :invalid_parameters
end

it "returns an unsuccessful result if the interval specified is not recognised" do
parameters[:workers][:metadata][:every] = "invalid"

result = subject.perform
expect(result).to be_a Result
expect(result).not_to be_success
expect(result.error.code).to eq :invalid_parameters
end

it "creates supplier and associated workers" do
expect {
expect {
expect(subject.perform).to be_success
}.to change { SupplierRepository.count }.by(1)
}.to change { BackgroundWorkerRepository.count }.by(2)

supplier = SupplierRepository.last
workers = BackgroundWorkerRepository.for_supplier(supplier).to_a

expect(supplier.name).to eq "Supplier X"
expect(workers.size).to eq 2

worker = workers.first
expect(worker.type).to eq "metadata"
expect(worker.next_run_at).to be_nil
expect(worker.interval).to eq 24 * 60 * 60 # one day
expect(worker.status).to eq "idle"

worker = workers.last
expect(worker.type).to eq "availabilities"
expect(worker.next_run_at).to be_nil
expect(worker.interval).to eq 2 * 60 * 60 # two hours
expect(worker.status).to eq "idle"
end

it "updates changed data on consecutive runs" do
subject.perform

supplier = SupplierRepository.named("Supplier X")
workers = BackgroundWorkerRepository.for_supplier(supplier).to_a
metadata_worker = workers.find { |w| w.type == "metadata" }

expect(metadata_worker.interval).to eq 24 * 60 * 60

# updates metadata worker interval to every 2 days
parameters[:workers][:metadata][:every] = "2d"

expect {
subject.perform
}.not_to change { SupplierRepository.count }

metadata_worker = BackgroundWorkerRepository.find(metadata_worker.id)
expect(metadata_worker.interval).to eq 2 * 24 * 60 * 60 # 2 days
end

context "interval parsing" do
it "understands seconds notation" do
parameters[:workers][:availabilities][:every] = "10s"
subject.perform

supplier = SupplierRepository.named("Supplier X")
worker = BackgroundWorkerRepository.
for_supplier(supplier).
find { |w| w.type == "availabilities" }

expect(worker.interval).to eq 10
end

it "understands minutes notation" do
parameters[:workers][:availabilities][:every] = "10m"
subject.perform

supplier = SupplierRepository.named("Supplier X")
worker = BackgroundWorkerRepository.
for_supplier(supplier).
find { |w| w.type == "availabilities" }

expect(worker.interval).to eq 10 * 60
end
end
end
end