README

ArtifactType: npm package. Documentation: https://1dsdocs.azurewebsites.net/getting-started/javascript-getting_started.html Language: typescript Platform: various Stackoverflow: N/A Tags: 1DSJSSDK ms.reviewedAt: 02/21/2019 ms.reviewedBy: ramthi

Microsoft 1DS Web SDK Post Plugin

Description

1DS Web SDK Post Channel main functionality is to send data to OneCollector using POST, currently supporting XMLHttpRequest, fetch API and XDomainRequest.

npm

Package available here.

Basic Usage

Setup

import { AppInsightsCore, IExtendedConfiguration } from '@microsoft/1ds-core-js';
import { PostChannel, IChannelConfiguration } from '@microsoft/1ds-post-js';

var appInsightsCore: AppInsightsCore = new AppInsightsCore();
var postChannel: PostChannel = new PostChannel();
var coreConfig: IExtendedConfiguration = {
      instrumentationKey: "YOUR_TENANT_KEY",
      extensions: [
        postChannel
      ],
      extensionConfig: []
};
var postChannelConfig: IChannelConfiguration = {
    eventsLimitInMem: 50
};
coreConfig.extensionConfig[postChannel.identifier] = postChannelConfig;
//Initialize SDK
appInsightsCore.initialize(coreConfig, []);

Configuration

IChannelConfiguration

Config	Description	Type
eventsLimitInMem	The number of events that can be kept in memory before the SDK starts to drop events. By default, this is 10,000.	number
immediateEventLimit	[Optional] Sets the maximum number of immediate latency events that will be cached in memory before the SDK starts to drop other immediate events only, does not drop normal and real time latency events as immediate events have their own internal queue. Under normal situations immediate events are scheduled to be sent in the next Javascript execution cycle, so the typically number of immediate events is small (~1), the only time more than one event may be present is when the channel is paused or immediate send is disabled (via manual transmit profile). By default max number of events is 500 and the default transmit time is 0ms. Added in v3.1.1	number
autoFlushEventsLimit	[Optional] If defined, once this number of events has been queued the system perform a flush() to send the queued events without waiting for the normal schedule timers. Default is undefined	number
httpXHROverride	The HTTP override that should be used to send requests, request properties and headers should be added to the request to maintain correct functionality with other plugins.	IXHROverride
overrideInstrumentationKey	Override for Instrumentation key.	string
overrideEndpointUrl	Override for Endpoint where telemetry data is sent.	string
disableTelemetry	The master off switch. Do not send any data if set to TRUE.	boolean
ignoreMc1Ms0CookieProcessing	MC1 and MSFPC cookies will not be provided. Default is false.	boolean
payloadPreprocessor	POST channel preprocessing function. Can be used to gzip the payload (and set appropriate HTTP headers) before transmission.	Function
payloadListener	POST channel function hook to listen to events being sent, called after the batched events have been committed to be sent. Also used by Remote DDV Channel to send requests.	Function
disableEventTimings	[Optional] By default additional timing metrics details are added to each event after they are sent to allow you to review how long it took to create serialized request. As not all users require this level of detail and it's now possible to get the same metrics via the IPerfManager and IPerfEvent, so you can now disabled this previous level. Default value is false to retain the previous behavior, if you are not using these metrics and performance is a concern then it is recommended to set this value to true.	boolean
valueSanitizer	[Optional] The value sanitizer to use while constructing the envelope, if not provided the default sanitizeProperty() method is called to validate and convert the fields for serialization. The path / fields names are based on the format of the envelope (serialized object) as defined via the Common Schema 4.0 specification.	IValueSanitizer
stringifyObjects	[Optional] During serialization, when an object is identified should the object serialized by true => JSON.stringify(theObject); otherwise theObject.toString(). Defaults to false.	boolean
enableCompoundKey	[Optional] Enables support for objects with compound keys which indirectly represent an object eg. event: { "somedata.embeddedvalue": 123 } where the "key" of the object contains a "." as part of it's name. Defaults to false.	boolean
disableOptimizeObj	[Optional] Switch to disable the v8 `optimizeObject()` calls used to provide better serialization performance. Defaults to false.	boolean
avoidOptions _{(Since 3.1.10+ === true*)}	[Optional] Avoid adding request headers to the outgoing request that would cause a pre-flight (OPTIONS) request to be sent for each request. _{* This currently defaults to true but will be changed once the collector handles additional settings to allow the browser to better cache any previous OPTIONS response, at which point some of the current dynamic values sent on the query string will be moved to a header.}	boolean

IXHROverride

Config	Description	Type
sendPOST	This method sends data to the specified URI using a POST request. If sync is true then the request is sent synchronously. The oncomplete function should always be called after the request is completed (either successfully or timed out or failed due to errors).	function

Payload Preprocessors

interface IPayloadData {
    urlString: string;
    data: Uint8Array | string;
    headers?: { [name: string]: string };
}

type PayloadPreprocessorFunction = (payload: IPayloadData, callback: (modifiedBuffer: IPayloadData) => void) => void;

To perform some preprocessing operation on your payload before it is sent over the wire, you can supply the POST channel config with a payloadPreprocessor function. A typical usage of it would be to gzip your payloads.

const zlib = require('zlib');
const gzipFn: PayloadPreprocessorFunction = (payload: IPayloadData, cb) => {
  zlib.gzip(payload.data, (err, dataToSend) => {
    if (err) return cb(payload); // send original payload on error
    const payloadToSend = {
      ...payload,
      headers: { ...payload.headers, 'Content-Encoding': 'gzip' };
      data: dataToSend,
    };
    cb(payloadToSend);
  });
}

XHR Override sample using node.js Https module

const oneDs = require('@microsoft/1ds-analytics-js');
const https = require('https');

// XHR override using node.js https module
var customHttpXHROverride= {
  sendPOST: (payload, oncomplete) => {
    var options = {
      method: 'POST',
      headers: {
        ...payload.headers,
        "Content-Type": "application/json",
        "Content-Length": Buffer.byteLength(payload.data)
      }
    };
    const req = https.request(payload.urlString, options, res => {
      res.on('data', function (responseData) {
        oncomplete(res.statusCode, res.headers, responseData.toString());
      });
    });
    req.write(payload.data);
    req.end();
  }
};
var postChannelConfig = {
    httpXHROverride: customHttpXHROverride
};

IValueSanitizer Paths / Fields which are excluded

To ensure that the service can continue to accept events that are sent from the SDK, some paths and fields are explicitly blocked and are never processed via any configured valueSanitizer. These fields are blocked at the serialization level and cannot be overridden.

The path / fields names are based on the format of the envelope (serialized object) as defined via the Common Schema 4.0 specification, so the path / field names used for the IValueSanitizer are based on how the data is serialized to the service (CS 4.0 location) and not specifically the location on the event object you pass into the track methods (unless they are the same).

The currently configured set of fields include

Excluded Part A fields

All direct top-level fields of the envelope, this includes "ver"; "name"; "time" and "iKey". see Common Schema 4.0 - Part A for all defined fields. Note: This exclusion does does not match sub-keys (objects) like "ext", "data" and "properties".
All fields of the web extension, this includes all fields and sub keys of "ext.web" (example fields include "ext.web.browser"; "ext.web.domain"; "ext.web.consentDetails"). see Common Schema 4.0 - Part A Extension - web for the complete set of defined fields.
All fields and paths of the metadata extension, this includes all fields and sub keys of "ext.metadata". see Common Schema 4.0 - Part A Extension - metadata for the complete set of defined fields.

Synchronous Events

By default events are batched and sent asynchronously, there are times when you want to send events immediately during the same JavaScript execution cycle.

To support this you can set the sync property on an event to tell the PostChannel to skip the normal event batching and send this event now within it's own outbound connection. Because each sync event will cause a new request / connection (per event) you should use this approach SPARINGLY to avoid creating an excessive number of requests from the users browser which may have a negative impact on their experience.

Note: If the initial synchronous request fails (not the normal case) any sync event will be queued for resending as an asynchronous batch (unless an "unload" event has been detected).

Supported Sync values

Supported event sync values to cause an event to be sent immediately during the same JavaScript execution cycle.

In the case of the sendBeacon() and fetch() [with keepalive] (SyncFetch) both of these API's have a maximum payload size defined as 64Kb, as such if the size of the serialized (JSON) events are larger than this the events will be dropped as there is no safe way to send the event.

Name	Value	Description
Batched	undefined, false, 0	This is the default situation and will cause the event to be Batched and sent asynchronously.
Synchronous	true, 1	Attempt to send the request using the default synchronous method. This will use the first available transport option: - httpXHROverride; -XMLHttpRequest (with sync flag); -fetch() [with keepalive] (Since v3.1.3); -sendBeacon() (Since v3.1.3)
SendBeacon	2	(Since v3.1.3) Attempt to send the event synchronously with a preference for the sendBeacon() API. This will use the first available transport option: -sendBeacon(); -fetch() [with keepalive]; -XMLHttpRequest (with sync flag); -httpXHROverride
SyncFetch	3	(Since v3.1.3) Attempt to send the event synchronously with a preference for the fetch() API with the keepalive flag, the SDK checks to ensure that the fetch() implementation supports the 'keepalive' flag and if not it will skip this transport option. This will use the first available transport option: -fetch() [with keepalive]; -sendBeacon(); -XMLHttpRequest (with sync flag); -httpXHROverride

The named values are available in TypeScript via the EventSendType const enum class since v3.1.3.

Note: The SDK explicitly checks for keepalive support for fetch() via the Request class, so if not available this transport will not be used. Browsers natively supporting fetch() keepalive, as such any polyfill would also need to support this flag for it to be used.

Special sendBeacon() Note

As the sendBeacon() API does not provide any response handling, any events sent via this method are considered to have been successfully sent, even if the browser fails to send the requests. The only exception is that if the browser returns false indicating that it can't accept the sendBeacon() request, in this case the SDK will send a dropped event notification.

When to use 'sync' events

Events that cause a page navigation can cause a race condition that could result in either event loss or duplication. This happens when any previously batched events have been sent (the request is in-flight) and the browser subsequently cancels the request before a response is processed AND it also triggers the JavaScript "cancel" or "abort" event, normally only during the unload process.

The SDK listens to all supported "unload" events (unload, beforeunload and pagehide) and when any one of these are detected it will immediately send all (unsent) batched events via the "SendBeacon" (and fallback) methods above. This successfully mitigates the event loss case above, but it can compound the event duplication case for some scenarios as the SDK works to ensure that all events are sent and acknowledged.

This situation has become more prevalent with the enforcement by modern browsers to disallow, cancel or abort the usage of synchronous Xhr requests during page "unload" event.

There are effectively 2 scenarios where you should consider adding the sync property to an event (using the asynchronous SendBeacon and SyncFetch values) to remove the possibility of event duplication (the WebAnalytics extension already handles these cases).

You want to send your own event during the page "unload" events and you have hooked the "unload" events yourself (before) initializing the SDK.

When you listen to the events "before" the SDK initializes, there is a small window of time where any "batched" event may get sent (and therefore become in-flight) before the SDK receives and processes the "unload" events, thus the potential race condition.
If you attach to the "unload" events "after" the SDK is initialized, it will now (since v3.1.3) automatically convert all received event(s) into "sync" (SendBeacon) events to ensure delivery.

You want to send you own telemetry event(s) based on some user action after which a page-navigation immediately occurs, either via an anchor <a /> containing a href and letting the event bubble or by directly causing a navigation via a form Post or location change.

This is because because the href / post / location change will eventually cause an "unload" event to occur and therefore (potentially) any outbound (in-flight) requests may get canceled and cause event to be duplicated.
While this situation can occur with any other batched events, it is more likely to occur when events are created during these know situations that are known to directly trigger the "unload" cycle.

API documentation

https://1dsdocs.azurewebsites.net/api/webSDK/chnl/post/3.0/f/index.html

Sample App

React sample App.

React Native sample App.

Learn More

You can learn more in 1DS First party getting started.

Data Collection

The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft's privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.

To turn off sending telemetry to Microsoft, ensure that the POST channel is not configured in the extensions. See below configuration for example:

var coreConfig: IExtendedConfiguration = {
      instrumentationKey: "YOUR_TENANT_KEY",
      extensions: [
        postChannel  // << REMOVE THIS EXTENSION TO STOP SENDING TELEMETRY TO MICROSOFT
      ],
      extensionConfig: []
};

License

MIT

Usage no npm install needed!