Home
(this user guide is under construction...)
see: Story Of the Tratt
The purpose of the "tratt-api" library is to help verifying that a tracking system is working as expected on an end-to-end level.
When a user interacts with a system, e.g. makes a purchase in an app or clicks a button on a webpage, one or multiple events are logged to a server. These logs on the server can later be used for data analysis. This logging mechanism is referred by King as tracking. The "tratt-api" is a regular java library API and can for example be used with any test automation framework, (e.g. JUnit, TestNG) or serve as a backend for a frontend GUI.
Sequences of expected events, aka CheckPoints, are specified using the Tratt Definition Language (TDL). A TDL is written using an easy-to-understand JSON format and serves as validation rules, see further below for TDL examples and in depth description of the TDL.
Starting the tratt-api will asynchronously start monitoring the event source. Events are processed and validated according to the CheckPoints in the TDL. Tratt-api does not put any constraints on how the events are triggered (For example events can be triggered manually, or from an automatic test). The tests will be successful upon fulfillment of checkPoints' criteria or fail if checkPoints are not fulfilled, or upon timeout. The below image and code illustrates this in practice.
The above illustration looks like this in code:
import static java.util.concurrent.TimeUnit.SECONDS;
import org.junit.Test;
import com.king.tratt.StartedEventProcessor;
import com.king.tratt.Tratt;
import com.king.tratt.tdl.Tdl;
public class Example {
@Test
public void canProcessEventsAccordingToTdl() throws Exception {
// Configure Tratt-api
StartedEventProcessor started = Tratt.newEventProcessorBuilder()
.setTimeout(5, SECONDS)
.addEventIterator(new MyEventIterator())
.setEventMetaDataFatory(new MyEventMetaDataFactory())
.setValueFactory(new MyValueFactory())
.addTdls(Tdl.fromPath("classpath:com/king/tratt/acceptance.tdl"))
.addVariable("userId", 12345)
// start monitoring the event source asynchronously
.start();
triggerYourEventsHere();
// then wait until fulfillment of checkPoints' criteria,
// or throws exception upon CheckPoint Failure, or timeout occurs.
started.awaitSuccess();
}
private void triggerYourEventsHere() {
// do things here that will trigger the expected events.
}
}Contributions are highly welcomed :) Both internal and external contributors are expected to use pull requests when contributing. External contributors need to submit (only once) a Contributor License Agreement. For more info see CONTRIBUTING.md
We believe it's the public communication/discussions that drives an open source community (i.e. avoid private discussions for the reason described here).
Use the project issues tracker for all communications and label your issue accordingly. For example if you get stuck and need help with anything, open a new issue and apply the "help wanted" label.
Maven artifact
<dependency>
<groupId>com.king.tratt</groupId>
<artifactId>tratt-api</artifactId>
<version>0.1.0</version>
</dependency>
From the project root folder execute the following command:
> gradlew build
Make sure JAVA_HOME environment variable is set and pointing to a JDK (Java 8 or higher) and that gradlew has execution rights.
TBD when first release is made and published to Maven Central/JCenter.
TBD
It is assumed that an event has a unique name and a set of key/value pairs (a.k.a. fields) associated to it. The field values can be of types: long, string, boolean. An event name may optionally have an unique id associated with it. The different event types are collectively known as an Event Structure.
This event structure consists of two different types of events:
| Event id* | Event Name | field1 | field2 | field3 | field4 | field5 |
|---|---|---|---|---|---|---|
| 100 | GameStart | userId (string) | level (long) | timestamp (long) | ||
| 200 | GameEnd | userId (string) | level (long) | timestamp (long) | score (long) | endReason (string) |
(* The Event Id is optional, if omitted the event name should be used as id).
The Tratt Definition Language is used to:
- define sequences of tracking events in an easy-to-understand JSON format
- analyze the occurrence of these sequences in event data streams
Below is an example how to create a simple TDL file that can detect a sequence of events. We will go through the details below.
{
"comment": "Comment describing the purpose of this TDL",
"variables": ["userId=alex"],
"sequences": [{
"type": "CONTAINER",
"name": "SequenceA",
"sequenceMaxTime": "PT60S",
"checkPoints": [{
"eventType": "GameStart",
"match": "userId==$userId",
"validate": "level==2 && timestamp>=0",
"set": ["GS_level=level", "GS_timestamp=timestamp"]
}, {
"eventType": "GameEnd",
"match": "userId==$userId",
"validate": "level==GS_level && timestamp>=GS_timestamp && score==123 && endReason=='some-reason'"
}]
}]
}A sequence of events is defined as a list of CheckPoints. The CheckPoint has the format:
{
"eventType": <EventName>,
"match": <match expression>,
"validate": <match expression>,
"set": [<variable name>=<Event field | numeric constant | string constant>, ...]
}
where:
- eventType is the event name as defined in your event structure.
- match is used to filter the exact event to validate. For example, there might be multiple events of same type but from different devices in your event data stream. Use a match criteria to filter the events from the device under test only. See Using match expressions below for syntax.
- validate is used to validate field values on the event matched by above filter. See Using match expressions below for syntax.
- set is used to store field values from this CheckPoint to be used in other CheckPoints within the same Sequence. See Using setters below for syntax.
As an example (using the example event structure above), lets say we want to create a CheckPoint that matches GameStart event from user "alex" and validates the timestamp field to be greater than 0:
{
"eventType": "GameStart",
"match": "userId=='alex'",
"validate": "timestamp > 0"
}
A number of CheckPoints can be put in a list and contained in a Sequence.
"sequences": [{
"type": "CONTAINER",
"name": "GamePlay",
"sequenceMaxTime": "pt15m",
"checkPoints": [{
"eventType": "GameStart",
"match": "userId=='alex'",
"validate": "timestamp>0"
}, {
"eventType": "GameStart",
"match": "userId=='alex'",
"validate": "timestamp>0 && score>=1 && endReason!=''"
}]
}]where:
- type is the sequence type to use. Currently only "CONTAINER" type is supported, which expects the events in any order.
- name is a unique name for the sequence. Can be used to retrieve the result of this sequence from the tratt-api.
- sequenceMaxTime is the maximum time for the sequence to live, starting from the first matched event in the sequence. The format is ISO 8106. ( "pt15m" is interpreted as 15 minutes. "pt30s" as 30 seconds and so on.). NOTE! The timeout is calculated using the timestamp given by the actual event, so it does not necessary needs to be "real time".
Match expressions allows you to test the fields of an event in a CheckPoint. The match expression format is described in detail here. Match expressions can be used in the:
- "match" section of a CheckPoint.
- "validate" section of a CheckPoint.
(In the examples below we will use the "match" section for example purposes).
Two values can be compared like this:
"match":"value1 <operator> value2"
Where, <operator> is one of the following:
| Operator | Meaning |
|---|---|
| && | value1 AND value2 are true |
| || | value1 OR value2 is true |
| == | equals |
| != | not equal |
| < | less than |
| > | greater than |
| <= | less or equal |
| >= | greater or equal |
Values can be of the following types:
| Type | Comment |
|---|---|
| Event field | A field in the event used in the current checkPoint. Useing the example Event Structure above, if the eventType is GameStart, the fields userId, level, timestamp are valid. |
| Sequence-local variable | A sequence-local variable that was created using set in a previous checkPoint. See Using setters. |
| TDL file global variable | A global variable defined using the "variables" section at the top-level of the TDL file. see Using global variables. |
| Result of comparison | The result of a sub-expression in parentheses. |
| Numerical constant | A number, e.g. 1234 |
| String constant | A string in single-quotes, e.g. 'some text' |
Parentheses can be used to create more complex expressions:
"match":"value1 <operator> (value2 <operator> value3)"
Values that are representable as numbers can be used in arithmetical operations, for example:
"match":"value1 == (value2 + value3)"
The supported operators are:
| Operator | |
|---|---|
| + | |
| - | |
| * | |
| / | |
| % (modulo) |
There are special functions that can be used in the "validate" section:
| Syntax | comment |
|---|---|
jsonfield(<json-field-name>, <value>) |
Use this when a field value is a JSON string and you need to validate a specific field within the JSON string. |
substr(start,end,<value>) |
extract a substring from an event field value. |
concat(<value>,<value>, ...) |
use this to concat two ore more values. |
split(<value>, delimiter, index) |
Use this to verify elements in a coma separated string. |
Here are some examples of match expressions:
"match":"0"
"match":"!0"
"match":"score==9"
"validate":"score==9 && timestamp> 1"
"validate":"(score==9 && timestamp > 1) || userId < (1 + 2)"
"match":"!(score==9 && timestamp > 1) || userId < 1"
"validate": "jsonfield('id', aField) == $myVariable"
"validate":"longId == concat(fieldA, fieldB)"
"validate":"800 == split(aField, ',', 2)"
Setters provide the ability to store a value during a sequence. You set a variable in one CheckPoint and then it can be used in a match expression in another CheckPoint. The stored values has a sequence-local scope.
The setter has the form
"set": [<variable name>=<Event field | numeric constant | string constant>, <...=...>, <...=...>, ...]
Some examples:
"set":["var1=1", "var2='some string'", "var3=score", "var4=timestamp"]
See above TDL example where the first CheckPoint sets variables and the second CheckPoint uses the variables to validate the event.
Global variables can be defined in the "variables" section at the top level of the TDL, in the form:
"variables": [<variable name>=<value>, <...=...>, <...=...>, ...]
Variables can be accessed in match expressions, using the $ prefix. See above TDL example where the variable userId is defined in "variables" section and used as $userId in a match expression to validate the event.
Tratt-api depends on services provided by the client. Those services are explained below. Furthermore, it is also explained how to configure the tratt-api to use those services.
The EventIterator service is the layer that decouples the actual event source from the tratt-api. The client implementation is responsible for consuming the events (from its source) and wrap each event in an Event adapter and provide it to the tratt-api.
See the EventIterator javadoc for its contract.
See unit test implementation for a working example: TestFromFileEventIterator
The client should wrap their event in an Event adapter, in order to provide event id and the timestamp to the tratt-api.
See the Event javadoc for its contract.
See unit test implementation for a working example: TestEvent
The EventMetaDataFactory service is used in the initial stage of the execution life cycle, when parsing the TDL file. Its purpose is to create an EventMetaData instance for each CheckPoint's eventType section in the TDL file.
If, for some reason (for example misspelling in the TDL file), the eventType is not known in the client event structure null should be returned (or use the convenient default method EventMetaDataFactory.unknown()). This will cause the tratt-api to fail in an early stage and throw an exception pinpointing the problem in the TDL.
See the EventMetaDataFactory javadoc for its contract.
See unit test implementation for a working example: TestEventMetaDataFactory
This represents the meta data of an Event, currently the name and id of an Event. Id is optional and can be same as the name if not used. The purpose of EventMetaData is to map the CheckPoint's eventType field in the TDL to an id.
See the EventMetaData javadoc for its contract.
See unit test implementation for a working example: TestEventMetaData
The ValueFactory service is used in the initial stage of the execution life cycle, when parsing the TDL file. Its purpose is to create a Value instance for each event field in the CheckPoint's match expressions in the TDL file (remember match expressions are used in both match and validate sections of a CheckPoint).
Note that the ValueFactory.getValue(String eventName, String node) method will be fed with all nodes in the match expressions and it is critical that it only returns a Value instance when the node is a known event field for the given eventName. Otherwise null shall be returned (or use the convenient default method ValueFactory.unknown()).
The tratt-api will internally use this behavior to detect errors in the TDL and fail in an early stage and throw an exception pinpointing the problem in the TDL.
The Value instance is feed with an Event from the event data stream (through Value.get(Event, Context) method). The responsibility of a Value instance is to know how to extract a value for a specific field from an event and parse that value to correct data type (supported types are: long, string, boolean).
See the Value javadoc for its contract.
See unit test implementations for working examples: BooleanEventValue IdEventValue LongEventValue StringEventValue TimeStampEventValue
There are a few different ways to configure the tratt-api with the client defined services. The easiest way is to set the services using the setter methods on the EventProcessorBuilder class, as shown in below example:
// Configure Tratt-api
StartedEventProcessor started = Tratt.newEventProcessorBuilder()
.addEventIterator(new MyEventIterator()) // add your EventIterators. Multiple EventIterators can be added.
.setValueFactory(new MyValueFactory()) // set your ValueFactory service
.setEventMetaDataFatory(new MyEventMetaDataFactory()) // set your EventMetaDataFactory Service
// start monitoring the event source asynchronously
.start();This can however be cumbersome, as you have to do it each time a StartedEventProcessor is built and you probably want the same client defined services all the time. See below for a solution to this.
ApiConfigurationProvider interface collects all client defined services into one interface.
public interface ApiConfigurationProvider {
ValueFactory valueFactory();
EventMetaDataFactory metaDataFactory();
}and can be set with this EventProcessorBuilder.setApiConfigurationProvider(...) method as shown below.
// Configure Tratt-api
StartedEventProcessor started = Tratt.newEventProcessorBuilder()
.setApiConfigurationProvider(new MyApiConfigurationProvider())
.addEventIterator(new MyEventIterator())
// start monitoring the event source asynchronously
.start();or you use java's ServiceLoadermechanism to set it. See below.
See Javadoc for ServiceLoader here which describes how to use it. Basically you need to add the following folders/file in your resources folder.
META-INF/services/com.king.tratt.spi.ApiConfigurationProvider
and the content should be the fully qualified name of your class that implements ApiConfigurationProvider.
Now your tratt-api configuration may look something like this:
// Configure Tratt-api
StartedEventProcessor started = Tratt.newEventProcessorBuilder()
.addEventIterator(new MyEventIterator())
// start monitoring the event source asynchronously
.start();TBD
TBD
answers to frequently asked questions in the github issues will be posted here.