Introduction

One of the MXP over UDP key concepts is to have a small set of UDP messages concentrated on connectivity, transport and distributed virtual environment scene graph synchronization which form the core of the protocol. All application specific information is transmitted as payload over the core UDP messages encoded in MXMP format. MXMP is designed to be XML but defined only to skeletal level as it is application specific in nature. Another key concept is to multiplex all the data (except binary assets) through one UDP port to simplify the session handling and networking like firewall and nat penetration.

Each UDP packet may contain one or more frames. Most messages will have length of just one frame.

All coordinates are always given in the coordinate system of the sending party.

Changes

1. Session Handling
   1. Incoming and outgoing packets have separate session ids for each connection. Session id is defined for each connection by sender. First response packet (ack) is paired to outgoing session by sent packet id replied in the ack.
2. PACKET_HEADER
   1. time field size fixed to 64 bits.
3. BUBBLE_FRAGMENT
   1. realtime field size fixed to 64 bits.
   2. Renamed bubble_server_address => bubble_address
   3. Renamed bubble_server_port => bubble_port
4. PROGRAM_FRAMET
   1. program_name length from 40 to 25
5. JOIN_REQUEST
   1. Realtime field size fixed to 64 bits.
   2. participant_name => participant_identifier
   3. participant_passphrase => participant_secret
   4. Added avatar_id field which is the main avatar injected by client
   5. Changed location name size to 28 bytes.
   6. identity_provider_url length to 50
6. JOIN_RESPONSE
   1. Realtime field size fixed to 64 bits.
   2. asset_cache_provider_url to 50
7. LIST_BUBBLE_REQUEST
   1. changed list_type enumeration values to 0=hosted,1=linked and 2=connected

Next Version Tasks

1. Evaluate whether there should be a simplified version of modification request which sends only the application specific extension instead of combination of object fragment and application specific extension. This would lower the overhead when only application specific values are being changed.
2. Evaluate whether application extensions could be divided to several segments which could be cached on bubble for full synchronization requests. This would allow efficient partial updates to objects using segments.
3. Evaluate whether ObjectFragment should contain InventoryUrl as it is suggested that each object should have home inventory defined where object assets can be loaded from and where object is persisted.
4. Evaluate whether object should contain persistent for deducing if bubble should host the object persistently. In other words whether object should disappear automatically when participant disconnects.
5. Evaluate whether object should contain avatar flag to mark it as avatar object. As an optimization perception messages would be sent of objects which are inside perception range of avatar objects.
6. Evaluate whether separate inject and modify messages are needed as they seem to be handled identically on implementation.
7. Evaluate whether object perception range should in included to object fragment.
8. Evaluate whether collision event should be added. Collision event would contain both colliding object Ids. Collision is detected by primary bubbles of both objects. Owning participant is applying the collision to the object trajectory. If objects are in separate bubbles then collision events are sent twice. Only first one is responded to. There has to be cool down time window before reacting to new collision event.
9. Change action observation radius to perception range.
10. Change action source object id to object id.
11. Evaluate if action object id should be changed to object index.
12. Evaluate using primary bubble to denote the bubble where object was injected in contrast to secondary bubble where object is replicated.
13. Evaluate whether bubble should have veto on movement. In other words should all movement be done with request / response.
14. Evaluate whether disappearance event should use objectid instead of object index. In some situations delayed disappearance might not have objectindex available.
15. Consider global identification schemes. Global uuid register, best effort uuid generation with collision possibility and urls as ids. This has to be considered for participants, objects and assets.
16. Consider methods for space governance in distributed bubble cloud. Access limits to given space including movement and injection. Ability to inject or move to space normal object, exclusively space reserving object, persistent object, immobile non physics respecting object? Limits to allowed object dimensions? Does this require changes to protocol or can it be handled through extension fragments and contracts between linked bubbles?
17. Evaluate removing owner id from bubble. No real use case. Owners may or may not want to mark their ownership and plain id is not anyway enough for this without global guid register. If this is needed one possibility would be to have openid identity url. Might be better to have separate method for accessing bubble metadata from bubble http service.
18. Evaluate whether TypeId is required in object and who upkeeps the register.
19. Evaluate if extension fragment type should have longer field than 4 characters.
20. Evaluate whether messages containing N-fragments should declare the amount of fragments in some kind of header fragment.
21. Find standard binary format to present time stamp.
22. Find better name for sequence_id field. For example stream_id could be better as sequence_id is different for upstream and downstream packets.
23. Evaluate whether third channel is needed which would provide guaranteed ordered message delivery. Could this be implemented on message queue level without touching the packet transport layer?
24. Evaluate whether dynamic fragment sections should have leading byte containing number of fragments. This would make some parsing strategies easier.
25. Evaluate whether time stamps are needed on packet level.
26. Evaluate whether string field sizes should be changed from fixed to variable from 0 to defined maximum width. The actual field size in bytes would be defined by leading byte (string) or integer (text). This would decrease the serialized message sizes but make parsing more complicated as frame content would not be predefined anymore. This would in practice lead to message content agnostic multiplexing and demultiplexing of message data to packet frames via specialized stream. Performance penalty of byte array copies could be avoided by implementing input stream which would allow reading of bytes from separate byte arrays of packets. This would keep the packets reserved until all the frames have been read from the packet.
27. Evaluate possibility of specifying message structure definition language which would allow for code generation of message valuable objects. The same system could be used to define extension fragment serialization.

Encoding

Byte ordering

http://en.wikipedia.org/wiki/Endianness

Little-endian byte ordering is used, except for serializing UUIDs (see below)

Signed fixed point representation

Signed values are represented in the Two's complement signed representation.

http://en.wikipedia.org/wiki/Two's_complement

Floating Point Encoding

The IEEE Standard for Binary Floating-Point Arithmetic (IEEE 754) is used for floating point encoding.

http://en.wikipedia.org/wiki/IEEE_754-1985

Universally Unique Identifier Encoding

UUIDs are serialized in big-endian byte order, as per section 4.1.2 of the UUID specification.

http://www.ietf.org/rfc/rfc4122.txt

Types

double: signed 64 bit float

udouble: unsigned 64 bit float

float: signed 32 bit float

ufloat: unsigned 32 bit float

long: signed 64 bit integer

ulong: unsigned 64 bit integer

int: signed 32 bit integer

uint: unsigned 32 bit integer

short: signed 16 bit integer

ushort: unsigned 16 bit integer

string: UTF8 encoded string (see below for details)

byte: unsigned 8 bit integer

uuid: 16 byte Universally Unique Identifier

time: 64 bit timestamp. How to encode?

• Proposal: milliseconds since 1.1.2000 00:00 UTC

data: Binary data. If data consists of binary encoded primitive values then they should be encoded as defined here. Data may also contain for example XML-fragment.

String representation

Strings are represented in UTF-8 encoding, and with zero-termination.

Usually strings are represented in fixed with spaces. If a string fits into the allocated space, the remaining free space is padded out with zero characters as necessary. If the UTF-8 representation of the string would be longer than the available space, it is truncated so that the last complete UTF-8 sequence and the zero-termination still fit into the space available.

URL Format

Bubbles can be referenced with the following URI type:

mxp://<address>:<port>/<bubble-name>/<location>

mxp://<address>:<port>/<bubble-id>/<location>

or with default values:

mxp://<address>/

mxp://<address>:<port>/

mxp://<address>:<port>/<bubble-name>

Sockets and Connectivity

Socket Handling

Each peer has one socket which is used to send and receive all information. Incoming packets are tied to connection based on remote address, port and incoming session id. Peers are divided to clients, servers and hubs. Clients and servers handle participant to bubble communication. Hubs are responsible for bubble to bubble communication.

Connection

Connection is initiated by join request or attach request. First response packet (ack) is paired to outgoing session by sent packet id replied in the ack. New connection request is always responded first by ack. Incoming and outgoing packets have separate session ids for each connection. Session id is defined for each connection by sender. Session id is unique for senders address and port combination.

Disconnection

Disconnect can occur due to leave or detach request which has been successfully responded with response. Disconnect can also occur due to timeout or protocol error.

Data Structures

Packet Structure (maximum 1452 bytes)

UDP packet data length is maximum of (1500-48(IPv6))=1452 bytes which consists of header bytes and as many frames as fit in the remaining bytes.

PACKET

packet header (18 bytes) - message frames (mxa 1434 bytes)

PACKET HEADER

4  : uint    : session_id /* Upstream session id. */
4  : uint    : packet id /* Id of the packet. */
8  : time    : first send time /* Time the packet was sent for the first time. */
1  : byte    : quaranteed /* Whether this is guaranteed and ACK is expected. */
1  : byte    : resend count /* Count of resends. */

Message Structure (maximum 265 bytes)

Largest message data sizes are limited to by maximum frame count and frame data size to ~15Mbytes. Could this protocol be used for transmitting larger amounts of data on background? If so another message type can be specified for large file transfers where each message transmit part of the file.

MESSAGE FRAME

frame header (10 bytes) - frame data (maximum 255 bytes)

FRAME HEADER

1  : byte    : type /* Type index from message type table. */
4  : uint    : message_id /* Message id assigned by the sender. */
2  : short   : frame_count /* Number of frames in this message. */
2  : short   : frame_index /* Index of the frame. */
1  : byte    : frame_data_size (0-255)

Reusable Fragments

RESPONSE FRAGMENT

4  : uint    : request_message_id
1  : byte    : failure_code // 0 == success

PROGRAM FRAGMENT

25 : string  : program_name
1  : byte    : program_major_version
1  : byte    : program_minor_version
1  : byte    : protocol_major_version
1  : byte    : protocol_minor_version
4  : uint    : protocol_source_revision

OBJECT FRAGMENT

16 : uuid     : object_id /* Universally unique identifier of the object. */
4  : uint     : object_index /* Bubble object index.. */
16 : uuid     : type_id /* Type id of the object. */
16 : uuid     : parent_object_id /* Identifier of the parent object. */
20 : string   : object_name
20 : string   : type_name
16 : uuid     : owner_id /* Owner participant id. */ 
12 : vector3f : location
12 : vector3f : velocity
12 : vector3f : acceleration
16 : vector4f : orientation
16 : vector4f : angular_velocity
16 : vector4f : angular_acceleration
4  : float    : bounding_sphere_radius
4  : float    : mass
4  : string   : extension_dialect
1  : byte     : extension_dialect_major_version
1  : byte     : extension_dialect_minor_version
4  : uint     : extension_length
X  : data     : extension_data

INTERACTION FRAGMENT

20 : string   : interaction_name
16 : uuid     : source_participant_id
16 : uuid     : source_object_id
16 : uuid     : target_participant_id
16 : uuid     : target_object_id
4  : string   : extension_dialect
1  : byte     : extension_dialect_major_version
1  : byte     : extension_dialect_minor_version
4  : uint     : extension_length
X  : data     : extension_data

BUBBLE FRAGMENT

16 : uuid     : bubble_id
40 : string   : bubble_name
51 : string   : bubble_asset_cache_url /* Url for accessing binary assets. */
16 : uuid     : owner_id /* Participant who owns the bubble. */
40 : string   : bubble_address
4  : uint     : bubble_port
12 : vector3f : bubble_center /* Defined in receiver coordinate space. */
4  : float    : bubble_range
4  : float    : bubble_perception_range
8  : time     : bubble_realtime

Control Messages

ACKNOWLEDGE 1 (not quaranteed)

1-64 X 4  : uint    : acknowledged_packet_id /* Repeated 1-64 times */

KEEPALIVE 2

THROTTLE 3

4  : uint    : transfer_rate /* Bytes per second sender is ready to receive */

Challenge Messages

Challenge request and response can be used to validate type and version of peer implementation.

CHALLENGE REQUEST 4

64 : data     : challenge_request_data

CHALLENGE RESPONSE 5

64 : data     : challenge_response_data

Command Messages

Participant to Bubble Commands

JOIN REQUEST 10

16 : uuid     : bubble_id /* bubble id or empty id */
16 : uuid     : avatar_id /* id of the avatar. */
40 : string   : bubble_name /* bubble name or empty string */
28 : string   : location_name /* landing location or empty string */
32 : string   : participant_identifier /* participant identifier */
32 : string   : participant_secret /* passphrase or authentication token */
8  : time     : participant_realtime /* real time of participant */
50 : string   : identity_provider_url /* for example OpenId provider url */
PROGRAM FRAGMENT : client_program

JOIN RESPONSE 11

RESPONSE FRAGMENT
16 : uuid     : bubble_id
16 : uuid     : participant_id /* id of the participant or empty id. */
16 : uuid     : avatar_id /* id of the avatar or empty id. */
40 : string   : bubble_name
50 : string   : bubble_asset_cache_url /* Url for accessing binary assets. */
4  : float    : bubble_range
4  : float    : bubble_perception_range
4  : time     : bubble_realtime /* real time of bubble at connect time. */
PROGRAM FRAGMENT : server_program

LEAVE REQUEST 12

LEAVE RESPONSE 13

RESPONSE FRAGMENT

INJECT REQUEST 14

OBJECT FRAGMENT

INJECT RESPONSE 15

RESPONSE FRAGMENT

MODIFY REQUEST 16

OBJECT FRAGMENT

MODIFY RESPONSE 17

RESPONSE FRAGMENT

EJECT REQUEST 18

16 : uuid     : object_id

EJECT RESPONSE 19

RESPONSE FRAGMENT

INTERACT REQUEST 20

INTERACTION FRAGMENT: request

INTERACT RESPONSE 21

RESPONSE FRAGMENT
INTERACTION FRAGMENT: response

EXAMINE REQUEST 22

16 : uuid     : object_id
4  : uint     : object_index

Either object_id or object_index will be set to default value. (uuid default==empty uuid=="0000-000....")

EXAMINE RESPONSE 23

RESPONSE FRAGMENT
OBJECT FRAGMENT

Bubble to Bubble Commands

ATTACH REQUEST 30

16 : uuid     : target_bubble_id
BUBBLE FRAGMENT (source bubble) 
PROGRAM FRAGMENT (source bubble server) 

ATTACH RESPONSE 31

RESPONSE FRAGMENT 
BUBBLE FRAGMENT (target bubble) 
PROGRAM FRAGMENT (target bubble) 

DETACH REQUEST 32

DETACH RESPONSE 33

RESPONSE FRAGMENT 

HANDOVER REQUEST 34

16 : uuid     : source_bubble_id
16 : uuid     : target_bubble_id
OBJECT FRAGMENT

HANDOVER RESPONSE 35

RESPONSE FRAGMENT 

Common Commands

LIST BUBBLES REQUEST 25

1  : byte    : list_type /* [0=hosted,1=linked,2=connected] */

LIST BUBBLES RESPONSE 26

N X {
BUBBLE FRAGMENT
60  : padding    : 60 padding bytes (zero values) to fill the frame.
}

Each fragment is in separate frame with 60 bytes padding added.

Event Messages

Common Events

PERCEPTION EVENT 40: an object has been observed for the first time or modified

OBJECT FRAGMENT

DISAPPEARANCE EVENT 45: Object disappears for another reason than ejection.

4  : uint     : object_index

MOVEMENT EVENT 41 (not quaranteed)

4  : uint     : object_index
12 : vector3f : location
16 : vector4f : orientation

ACTION EVENT 60: Object acts in a way which does not affect other objects.

20 : string   : action_name
16 : uuid     : source_object_id
4  : float    : observation_radius
4  : string   : extension_dialect
1  : byte     : extension_dialect_major_version
1  : byte     : extension_dialect_minor_version
4  : uint     : extension_length
X  : data     : extension_data

HANDOVER EVENT 53: Object was handed over from one bubble to another

HANDOVER REQUEST

SYNCHRONIZATION BEGIN EVENT 70: Bubble is starting to send full set of perceptions from objects in range.

4  : uint     : object_count /* Approximate perception count. */

SYNCHRONIZATION END EVENT 71: End of synchronization.

Event Candidates

COMPRESSED MOVEMENT EVENT 42 (not quaranteed)

N X {
4  : uint     : object_index
6  : vector3f : location
4  : vector3b : orientation
} /* N=1-12 (One message can contain 1-17 movement perceptions) 

This event is optimized for social 3d environments with size of ~1000 meters and accuracy of 6 centimeters. Environments with similar accuracy requirements can use this message as well. This message is one third of the size of MOVEMENT EVENT.

compressed location = range * location / (bubble observation radius)

compressed velocity = range * velocity / (bubble observation radius)

compressed orientation = axis angle with unit vector with components scaled to +-max value. Angle is presented in +-180 degress scaled to +-max value.

Total size 14 bytes + ~1 byte from all headers per message => total of ~15 bytes per compressed movement message.