Introducing Sweagle parsersSweagle supports various types of parsers (exporters, validators, and templates). This article will focus on exporters. Sweagle parsers can be written in JavaScript or Python – this article will focus on javascript. Sweagle Parser Logic Parsers are a way to apply logic on top of the data that is stored in 1 or multiple configuration data sets (CDS), and calculate a response. The parser logic can be a combination of extraction and transformation in order to return a subset of the data in the structure which is readily consumable by the requester. The extract logic can, for instance, be to only return a few of the config data items (example: return all the values of all CDIs with the name “IP Address”), return a single value (example: return the “Admin User”, or return a subset of the config data (example: return all CDIs for the “Server2”).The transform logic can be to structure the config data in a different way compared to how it is stored in the data set. As an example, the same config data can be requested by Ansible in its typical inventory format while a monitoring tool requests a JSON with a different subset of the config data set. As config data typically resides in various sources and formats, it is often hard to handle config data and prepare it for usage in CI/CD pipelines and automation frameworks. The fact that the config data from the various sources is already consolidated into the Sweagle data model and made available as an “input” makes the writing of agnostic logic easy. The parser logic can also receive arguments as an input to further define how the logic works (example: call the parser with the argument that contains the server name, and the response of the parser will return the subset of data for the node with the “server name”). In other words, parsers are a way to make the stored config data ready for consumption in a structure and format that makes is readily usable for the requester. How do parsers of type “Exporter” work? Reusable: An exporter is a piece of reusable logic, written in JavaScript or Python, that is applied on top of the data set and it returns a result.JSON: The content of a config data set in the Sweagle data model is a multi-level nested JSON object which contains the “nodes” (in a path-based hierarchy) and the config data items (“keys” and “values”). It's also possible to access the metadata of the nodes and config data items.Input data sets: The exporter logic is applied on 1 or multiple config data sets – all config data sets are available in the parser logic in an array object called cds[index]. The primary assigned config data set is cds[0], and any additional ones have an incremental index number in the cds array.Arguments: The exporter logic can optionally receive arguments in the form of an object which can be parsed inside the logic. The arguments are available inside the logic through the “arg” variable. Most commonly the arg contains a JSON or YAML string which can be parsed.Logic: the exporter logic gives you unlimited capabilities to compare, check, transform, filter, concatenate, … Whatever handling of the data that is required to compute a response based upon the provided “inputs” (the cds[] array) and the provided “arguments” (the arg object) Response: The exporter response is an object which can be automatically returned in the requested format (JSON, YAML, XML, properties, in, or plain text format). Setup Creating a data model The Sweagle data model allows structuring the config data in various dimensions. Each data model can be set up and configured in an entirely specific and customizable way. It adopts a node-based hierarchical data model to which configData items are attached. Nodes are structured as a "root node" which is called a dimension. Every dimension is a tree structure with parent and child nodes. Sample data model In this example data (see screenshot), the Dev environment is defined as a configuration data set (CDS ). Dev has configData( ) named "Infra", which has three child nodes in this example. Infra has three configData items (CDI ) defined. Uploading data In order to perform changes to the data model, you must open a new data changeset or scope into an existing data changeset. You can open a new changeset by clicking the in the top-right menu. The data model can be created manually by adding a new dimension, click on the "Create Dimension" button. The child nodes with configData items can be created manually or you can right-click on the parent node and select option "Upload Data Here". This will take you to a screen (see image) which allows you to import data from a file or snippet. To get started with sample data models, a sample JSON file that can be uploaded is attached to this article (sample_data_model.txt). Approve data In order to approve the data changeset, you can click on the data changeset indicator (red box) in the top menu. That will show the full list of modifications that have been done in the scope of this changeset. You have the possibility to undo some changes and, if all are correct, approve the data changeset. Once the changeset is approved, all changes are applied to the primary data model and all other users will see the updated data model. Introducing the config data set, parser editor and parser logic Config data set The config data set contains all the config data for a given node that is flagged as config data set. Whenever a change happens on that node or to one of its child nodes, Sweagle will create a new snapshot of the config data and makes it available through its API. Example of config data set: Parser editor The parser editor is an embedded IDE in the Sweagle UI to write, test, and publish parsers. The parser editor can be found in Admin Panel > Config Data Parsers > Click on Create Config Data Parser. When creating and testing Config Data Parsers (Admin panel), you can select a config data set at the top. See the orange rectangle on the screenshot above. This selection will be available in the Code Editor as cds[0]. The Extra config data sets that you select will be available as cds[1], cds[2], … Underneath the Code Editor, at the bottom of the page, you can see the Config data sets value box, this allows you to select one of the config data sets and shows you the JSON notation of that CDS. This helps you understand whether the logic is doing what it is intended to do. Running the Parser The code editor allows you to write your own Javascript to manage what you want to export and how you want the data to be exported. By clicking the Run Parser button, the Javascript will be executed by the browser. The result is shown in the Export preview Emulated pane. Example: Parser logic The parser logic allows performing any kind of ETL operation on the config data that is provided in the input (the cds[index]) and based upon the value of the optional argument object (the arg variable). The attached pdf file Introduction to Sweagle parsers provides a list of sample parser logic.