{
    "id": "NetskopeWebTransactionsDataConnector",
    "title": "Netskope Web Transactions Data Connector",
    "publisher": "Netskope",
    "descriptionMarkdown": "The [Netskope Web Transactions](https://docs.netskope.com/en/netskope-help/data-security/transaction-events/netskope-transaction-events/) data connector provides the functionality of a docker image to pull the Netskope Web Transactions data from google pubsublite, process the data and ingest the processed data to Log Analytics. As part of this data connector two tables will be formed in Log Analytics, one for Web Transactions data and other for errors encountered during execution.\n\n\n For more details related to Web Transactions refer to the below documentation: \n 1. Netskope Web Transactions documentation: \n> https://docs.netskope.com/en/netskope-help/data-security/transaction-events/netskope-transaction-events/ \n",
    "graphQueries": [
        {
            "metricName": "Web Transactions data received",
            "legend": "NetskopeWebtxData_CL",
            "baseQuery": "NetskopeWebtxData_CL"
        },
        {
            "metricName": "Web Transactions Data Connector Errors",
            "legend": "NetskopeWebtxErrors_CL",
            "baseQuery": "NetskopeWebtxErrors_CL"
        }
    ],
    "sampleQueries": [
        {
            "description": "Netskope Web Transactions Data",
            "query": "NetskopeWebtxData_CL\n | sort by TimeGenerated desc"
        },
        {
            "description": "Netskope Web Transactions Data Connector Errors",
            "query": "NetskopeWebtxErrors_CL\n | sort by TimeGenerated desc"
        }
    ],
    "dataTypes": [
        {
            "name": "NetskopeWebtxData_CL",
            "lastDataReceivedQuery": "NetskopeWebtxData_CL\n            | summarize Time = max(TimeGenerated)\n            | where isnotempty(Time)"
        },
        {
            "name": "NetskopeWebtxErrors_CL",
            "lastDataReceivedQuery": "NetskopeWebtxErrors_CL\n            | summarize Time = max(TimeGenerated)\n            | where isnotempty(Time)"
        }
    ],
    "connectivityCriterias": [
        {
            "type": "IsConnectedQuery",
            "value": [
                "NetskopeWebtxData_CL\n            | summarize LastLogReceived = max(TimeGenerated)\n            | project IsConnected = LastLogReceived > ago(30d)"
            ]
        },
        {
            "type": "IsConnectedQuery",
            "value": [
                "NetskopeWebtxErrors_CL\n            | summarize LastLogReceived = max(TimeGenerated)\n            | project IsConnected = LastLogReceived > ago(30d)"
            ]
        }
    ],
    "availability": {
        "status": 1,
        "isPreview": true
    },
    "permissions": {
        "resourceProvider": [
            {
                "provider": "Microsoft.OperationalInsights/workspaces",
                "permissionsDisplayText": "read and write permissions on the workspace are required.",
                "providerDisplayName": "Workspace",
                "scope": "Workspace",
                "requiredPermissions": {
                    "write": true,
                    "read": true,
                    "delete": true
                }
            },
            {
                "provider": "Microsoft.OperationalInsights/workspaces/sharedKeys",
                "permissionsDisplayText": "read permissions to shared keys for the workspace are required. [See the documentation to learn more about workspace keys](https://docs.microsoft.com/azure/azure-monitor/platform/agent-windows#obtain-workspace-id-and-key).",
                "providerDisplayName": "Keys",
                "scope": "Workspace",
                "requiredPermissions": {
                    "action": true
                }
            }
        ],
        "customs": [
            {
                "name": "Azure Subscription",
                "description": "Azure Subscription with owner role is required to register an application in Microsoft Entra ID and assign role of contributor to app in resource group."
            },
            {
                "name": "Microsoft.Compute permissions",
                "description": "Read and write permissions to Azure VMs is required. [See the documentation to learn more about Azure VMs](https://learn.microsoft.com/azure/virtual-machines/overview)."
            },
            {
                "name": "TransactionEvents Credentials and Permissions",
                "description": "**Netskope Tenant** and **Netskope API Token** is required. [See the documentation to learn more about Transaction Events.](https://docs.netskope.com/en/netskope-help/data-security/transaction-events/netskope-transaction-events/)"
            },
            {
                "name": "Microsoft.Web/sites permissions",
                "description": "Read and write permissions to Azure Functions to create a Function App is required. [See the documentation to learn more about Azure Functions](https://docs.microsoft.com/azure/azure-functions/)."
            }
        ]
    },
    "instructionSteps": [
        {
            "title": "",
            "description": ">**NOTE:** This connector provides the functionality of ingesting Netskope Web Transactions data using a docker image to be deployed on a virtual machine (Either Azure VM/On Premise VM). Check the [Azure VM pricing page](https://azure.microsoft.com/pricing/details/virtual-machines/linux) for details."
        },
        {
            "title": "",
            "description": ">**(Optional Step)** Securely store workspace and API authorization key(s) or token(s) in Azure Key Vault. Azure Key Vault provides a secure mechanism to store and retrieve key values. [Follow these instructions](https://docs.microsoft.com/azure/app-service/app-service-key-vault-references) to use Azure Key Vault with an Azure Function App."
        },
        {
            "title": "",
            "description": "**STEP 1 - Steps to create/get Credentials for the Netskope account** \n\n Follow the steps in this section to create/get **Netskope Hostname** and **Netskope API Token**:\n 1. Login to your **Netskope Tenant** and go to the **Settings menu** on the left navigation bar.\n 2. Click on Tools and then **REST API v2**\n 3. Now, click on the new token button. Then it will ask for token name, expiration duration and the endpoints that you want to fetch data from.\n 5. Once that is done click the save button, the token will be generated. Copy the token and save at a secure place for further usage."
        },
        {
            "title": "",
            "description": "**STEP 2 - Choose one from the following two deployment options to deploy the docker based data connector to ingest Netskope Web Transactions data **\n\n>**IMPORTANT:** Before deploying Netskope data connector, have the Workspace ID and Workspace Primary Key (can be copied from the following) readily available, as well as the Netskope API Authorization Key(s) [Make sure the token has permissions for transaction events].",
            "instructions": [
                {
                    "parameters": {
                        "fillWith": [
                            "WorkspaceId"
                        ],
                        "label": "Workspace ID"
                    },
                    "type": "CopyableLabel"
                },
                {
                    "parameters": {
                        "fillWith": [
                            "PrimaryKey"
                        ],
                        "label": "Primary Key"
                    },
                    "type": "CopyableLabel"
                }
            ]
        },
        {
            "title": "Option 1 - Using Azure Resource Manager (ARM) Template to deploy VM [Recommended]",
            "description": "Using the ARM template deploy an Azure VM, install the prerequisites and start execution.\n\n1. Click the **Deploy to Azure** button below. \n\n\t[![Deploy To Azure](https://aka.ms/deploytoazurebutton)](https://aka.ms/sentinel-NetskopeV2WebTransactions-azuredeploy)\n2. Select the preferred **Subscription**, **Resource Group** and **Location**. \n3. Enter the below information : \n\t\tDocker Image Name (mgulledge/netskope-microsoft-sentinel-plugin:netskopewebtransactions)\n\t\tNetskope HostName \n\t\tNetskope API Token \n\t\tSeek Timestamp (The epoch timestamp that you want to seek the pubsublite pointer, can be left empty) \n\t\tWorkspace ID \n\t\tWorkspace Key \n\t\tBackoff Retry Count (The retry count for token related errors before restarting the execution.)  \n\t\tBackoff Sleep Time (Number of seconds to sleep before retrying) \n\t\tIdle Timeout (Number of seconds to wait for Web Transactions Data before restarting execution) \n\t\tVM Name \n\t\tAuthentication Type \n\t\tAdmin Password or Key \n\t\tDNS Label Prefix \n\t\tUbuntu OS Version \n\t\tLocation \n\t\tVM Size \n\t\tSubnet Name \n\t\tNetwork Security Group Name \n\t\tSecurity Type \n4. Click on **Review+Create**. \n5. Then after validation click on **Create** to deploy."
        },
        {
            "title": "Option 2 - Manual Deployment on previously created virtual machine",
            "description": "Use the following step-by-step instructions to deploy the docker based data connector manually on a previously created virtual machine."
        },
        {
            "title": "",
            "description": "**1. Install docker and pull docker Image**\n\n>**NOTE:** Make sure that the VM is linux based (preferably Ubuntu).\n\n1. Firstly you will need to [SSH into the virtual machine](https://learn.microsoft.com/azure/virtual-machines/linux-vm-connect?tabs=Linux).\n2. Now install [docker engine](https://docs.docker.com/engine/install/).\n3. Now pull the docker image from docker hub using the command: 'sudo docker pull mgulledge/netskope-microsoft-sentinel-plugin:netskopewebtransactions'.\n4. Now to run the docker image use the command: 'sudo docker run -it -v $(pwd)/docker_persistent_volume:/app mgulledge/netskope-microsoft-sentinel-plugin:netskopewebtransactions'. You can replace mgulledge/netskope-microsoft-sentinel-plugin:netskopewebtransactions with the image id. Here docker_persistent_volume is the name of the folder that would be created on the vm in which the files will get stored."
        },
        {
            "title": "",
            "description": "**2. Configure the Parameters**\n\n1. Once the docker image is running it will ask for the required parameters.\n2. Add each of the following application settings individually, with their respective values (case-sensitive): \n\t\tNetskope HostName \n\t\tNetskope API Token \n\t\tSeek Timestamp (The epoch timestamp that you want to seek the pubsublite pointer, can be left empty) \n\t\tWorkspace ID \n\t\tWorkspace Key \n\t\tBackoff Retry Count (The retry count for token related errors before restarting the execution.)  \n\t\tBackoff Sleep Time (Number of seconds to sleep before retrying) \n\t\tIdle Timeout (Number of seconds to wait for Web Transactions Data before restarting execution)\n3. Now the execution has started but is in interactive mode, so that shell cannot be stopped. To run it as a background process, stop the current execution by pressing Ctrl+C and then use the command: 'sudo docker run -d -v $(pwd)/docker_persistent_volume:/app mgulledge/netskope-microsoft-sentinel-plugin:netskopewebtransactions'."
        },
        {
            "title": "",
            "description": "**3. Stop the docker container**\n\n1. Use the command 'sudo docker container ps' to list the running docker containers. Note down your container id.\n2. Now stop the container using the command: 'sudo docker stop *<*container-id*>*'."
        }
    ]
}