Nav

Lab 2: Insight into Business Events

Overview

Insight is a tool that gives you in-depth visibility into business transactions and events on your Mule applications that you deploy through Runtime Manager. By tracking everything your data does in an application, Insight makes information searchable and helps you find and recover from any errors that occurred during message processing and replay your transactions instantly.

Insight helps you answer questions about your applications, such as:

  • What happened with a particular transaction or synchronization?

  • When did the transaction occur? How long did it take?

  • What was the result of a transaction?

  • If something went wrong during processing, at what point did the failure occur?

Using the Insight Dashboard, you can view events that occur within the flows that handle your business transactions. Further, you can drill down into event-related data to access information that can help you analyze the root cause of failures, isolate performance bottlenecks, and test for compliance with company procedures.

Transactions are logical groupings of related events, and often correspond to a business view of the system. Transactions map to flows.

Events are low-level details of a transaction. Events map to message processors and connectors. Events can also be exceptions and custom events. All events, except for custom events, are considered default events.

In this Lab you will enable Insight for your application, which will let you view and replay the default Business Events. Then you will add Custom Business Events to the application, redeploy it on CloudHub, and view your Custom Business Events within Insight.

The lab will consist of several steps:

Step 1: Enable default Business Events

To obtain detailed visibility into what is happening with your data, you can enable the tracking of default events for processes that have particular value to you. Determine which stages within a business transaction that you want to track, and enable tracking for those stages before deployment. Tracking all possible events is also an option, but you would have to spend more time at runtime filtering or querying to find the events you want to analyze.

Event tracking requires some processing and network overhead to aggregate and store the events that the Mule servers generate, so by default, tracking is not enabled for all connectors or message processors that support it. For everyday monitoring, it’s best to use the tools listed in the Monitoring section of our documentation.

Enabling tracking for default events is very simple. You just need to explicitly configure the scope for tracking the default events. You can configure the scope either at the flow level, or at the message processor and connector level. For this lab, you will enable Events at the Flow level.

  1. Start Anypoint Studio from the desktop icon Studio

  2. Open your full API implementation that you completed in Module 2, and click the title bar of the flow in the canvas to open the flow’s Properties Editor.

    module6 studio flow title
  3. In the Properties Editor, within the Business Events section, select true on the combo box Enable default events tracking

    module6 lab2 studio prop events

    Now we are going to set a Transaction Id to set an identifier for all tracked events pertaining to this flow so you will know which flow it is tracking and can easily search for it later.

  4. Go to the palette and search for Set Transaction Id

    module6 lab2 transactionId palette
  5. Drag and drop the component at the beginning of the flow

    module6 lab2 drag drop transactionId
  6. Click on the component.

    We are going to use a dataweave expression to represent the transaction id. You could use whatever you consider that will identity your flow.

  7. Press module6_lab2_fx_button button and add the following expression: 'get-products-search-' ++ uuid()

    module6 lab2 transaction id configure

    It’s good practice to customize the transaction ID such that the ID is unique for each transaction in your application. For example, you can use a Dataweave expression to set a unique ID based on a unique order ID extracted from a payload so that meaningful information, such as an order number, an employee identification number, or a shipment tracking number, is displayed for the ID of a transaction. This makes analysis and debugging easier and more intuitive at runtime. The transaction id can be set in any part of the flow. It doesn’t need to be set at the beginning.

  8. Don’t forget to Save your changes.

    You can do this the usual ways: via hotkey, File>Save, or the toolbar icon

Now that you’ve enabled flow-level Event Tracking, you will be able to use the Insight Dashboard to get visibility into things like when this flow is being processed and how long it took.

Step 2: Add a Custom Business Event

"Okay, flow-level detail is useful, but what I really care about is exposing more granular information about business metrics for this event."

Today is your lucky day, then. Mule also gives you the ability to track custom metadata in your flow using the custom business event message processor. Each custom business event can be configured to have multiple metadata properties associated with it.

Use custom events to track key process indicators and metrics, for example, "Total Order Amount" or "Tracking Number" to surface the high-level business activities in your flow. Like the Transaction ID, these can contain a dataweave expression to access payload data so you can expose meaningful data for Insight.

In this step, we will add a Custom Business Event, give it some parameterized KPIs to track, and then deploy your updated application to CloudHub.

  1. First we need to store the search query so we can use it later. In the Studio Palette, on the right, type variable to search for the Set Variable component

    module6 lab2 studio pal flowvar
    1. Drag the Set Variable component onto your get:/products/search:api-config flow. Place it as the first step in the flow by dropping it to the left of the "Run Search on Product API" component.

      module6 lab2 studio variable
    2. Click the Set Variable component to open its properties panel

    3. For the Name: field, enter "searchName". For the Value: field, enter

      #[attributes.queryParams.name]
      module6 lab2 studio set variable
  2. In the Studio Palette, on the right, type "event" to search for the Custom Business Event component

    module6 lab2 studio palette event
    1. Drag the Custom Business Event component onto your get:/products/search:api-config flow. Place it as the last step in the flow by dropping it to the right of the last Transform Message component.

      module6 lab2 studio cbe
    2. Click the Custom Business Event to open its properties panel

    3. Now you need to enter an Event Name that can be used in the search interface. Within the Custom Business Event properties panel, type a name for your Event. Event Name, of course, may be any valid Mule expression. Complete with the value Product Search Event.

      module6 lab2 studio cbe name
    4. Next, add two KPIs to track.

      To track the query string and what results are being presented, you can create one KPI each for the search and the returned results using the productName Flow Variable we stored earlier and a dataweave expression to access the results.

      KPI Name Expression/Value

      Search String

      #[vars.searchName]

      Search Results

      #[payload.name]

      module6 lab2 studio cbe kpi
    5. Save your changes

  3. Now that you’ve added a Custom Business Event and some KPIs to track, you are ready to deploy the new version of your API.

    1. Right-click your project and push your changes to CloudHub by selecting Anypoint Platform > Deploy to CloudHub

      module6 lab2 studio deploy menu
    2. Select the Insight tab

      module6 studio deploy insight
    3. Insight provides you with three options. Select the third option, "Metadata and Replay"

      • Disabled to not use Insight.

      • Metadata to simply track events.

      • Metadata and Replay to track events and enable the ability to replay a transaction from the console.

        • Insight’s Message Replay feature enables advanced error recovery. When you encounter a failed transaction (as displayed on Insight’s dashboard), you can diagnose the problem, fix the root issue, then replay the transaction that failed.

    4. Click on "Deploy Application" to overwrite the previous version of your application

    5. In the next window, you can use "Open in Browser" to launch Anypoint Runtime Manager, where you should see your application status says "Updating"

  4. Once your application has started, you are ready to send it some transactions to process. You can easily test your flow by using the API Console at at http://<username-omni-channel-api>.<region-id>.cloudhub.io/console. In this Lab’s example it would be http://fb-omni-channel-api-v1.us-e2.cloudhub.io/console

    module6 lab2 api console
    1. Your flow is called by the GET method of /products/search, so locate that and click the blue box GET

    2. This brings up a panel you can use to test your live API. Click "Try it"

      module6 lab2 api console tryit
    3. Enter a search string in the "name" field and then hit "GET"

      module6 lab2 api console name get

      Your API response will be displayed below, formatted in JSON.

      module6 lab2 api console response
    4. You can try different search strings to create additional Events. e.g., "max","sticker","T-shirt"

Step 3: Getting Insight into Business Events

After you enable Insight, the following events are tracked by default:

  • Data that passes through endpoints (inbound and outbound) in your Mule applications

  • Flow initiation and completion

  • Custom business events you embed in your Mule flows

Insight retains data from the last 100,000 Mule application events for up to 7 days, and it keeps a record (without the Mule message content) of all events from the last 30 days.

This step will show you how to view transactions, events, and metadata using the Insight feature.

  1. Open the management pane for your application in Anypoint Runtime Manager, then click Insight on the left

    module6 lab2 ap insight
  2. Click the expando arrow to expand one of the transactions

    Information about your Business Events is displayed at three levels

    1. the transaction level (an invocation of your flow)

    2. the event level (elements in your flow)

    3. the metadata level (KPI/Metrics recorded in your Custom Business Event)

      module6 lab2 insight flow default
  3. Insight’s Message Replay feature enables advanced error recovery. When you encounter a failed transaction (as displayed on Insight’s dashboard), you can diagnose the problem, fix the root issue, then replay the transaction that failed.

    1. To replay a transaction, click the replay icon replay next to the first event in the transaction

      Insight requests confirmation and then replays the transaction. And then displays a message that the Message was replayed.

      Insight displays replayed transactions immediately on the dashboard and in the Logs tab. The Message Replay option is only available on applications deployed to CloudHub. Logs for applications deployed to servers on-premises contain event information, but not message contents.

Summary

Event Tracking allows you to to view real-time information about Business Events in messages being processes by a specific flow. You can view message payloads before, during and after they are processed, as well as information about the message processors in the flow and message metadata. You can even replay a transaction after applying a change to your Mule application. This is a powerful capability, but with great power comes great responsibility (and, in this case, a performance hit.) Use it wisely.

In this lab, we completed these steps

If you’d like to learn more

Congratulations! You have completed Lab 2.

Please proceed to Lab 3