MMS 4.1.0 Merge Proposal

Owned by Jason Han
Last updated: Apr 12, 2022
2 min read

This is a proposal for the branch merge implementation in MMS 4.1.0

 

Merging of branches has always been a challenge for MMS, due to the fact that model data is stored as JSON. In the past, RAM limitations for unmarshalling JSON in Java has always caused issues, due to the fact that both JSON objects would have to be held in memory while determining a diff. This operation has traditionally been passed to clients (Cameo MDK) to implement. However, in the latest MMS4, there have been significantly improved strategies for streaming JSON and optimizing memory usage during processing exceptionally large JSON objects. Considering this, we propose that we apply these improvements in the implementation of the merging of references (branches and commits).

 

Proposed Endpoints:

GET /projects/{project_id}/refs/{parent_ref_id}/diff/{child_ref_id}

This endpoint should generate a JSON-Patch file with the differences between the given refs. parent_ref_id and child_ref_id should be either branch_id or commit_id.

 

PATCH /projects/{project_id}/refs/{parent_ref_id}/diff/{child_ref_id}

This endpoint should possibly accept the JSON-Patch file from the diff endpoint in the request body, or generate a new patch and apply the patch to the specified ref.

 

Concerns:

The first area of concern, which has caused issues in the past, is memory usage. MMS, as a result of handling model data as JSON, requires the loading of a complete JSON object (unmarshalling) in memory in order to manipulate and query model data. Since elements of branches are filtered and retrieved on request, this would generally require holding both refs in memory throughout the duration of the operation. However, with improvements in streaming capabilities in MMS 4, it may now be possible to keep only one ref in memory while streaming the other ref for comparison.

 

One other area of concern would be the enforcing of the application of the JSON-Patch file to the originally requested parent ref, as it may cause unexpected / incorrect elements in any other branch. It may be necessary to store diffed branches in MMS to enforce this, although this would not prevent all possible cases of incorrect application.

 

Yet another area of concern would be model validation of the patched ref. MMS currently has no mechanism to validate model data, and thus would not be able to validate that the end state of the patch operation is actually valid. As long as the patch is created correctly, this should not be too much of a concern, however, it could still result in the model being in an invalid state.