Monitoring active windows (for Windows systems)

One possible usage of streaming process mining could involve the monitoring of the current system. One possible way of achieving this goal is to monitor the window currently active (i.e., with the focus) as a proxy for the application being used by the user¹.

To achieve this goal it is possible to define a new XesSource which observes the windows currently active and, whenever there is a new window in focus, emits an event. To accomplish this goal, in the following we make use of the JNA (Java Native Access) library which gives us access to the native shared libraries of the operating system. To have access to the library we need, first of all, to include it in our Maven dependency:

<dependency>
    <groupId>net.java.dev.jna</groupId>
    <artifactId>jna</artifactId>
    <version>5.10.0</version>
</dependency>
<dependency>
    <groupId>net.java.dev.jna</groupId>
    <artifactId>jna-platform</artifactId>
    <version>5.10.0</version>
</dependency>

Once the library is include we can define the method that will return the name of the currently active window on the screen (this works and has been tested on Windows 10 Enterprise):

The documentation on the system calls used here can be found on the MSDN documentation (here, name="__codelineno-1-1">class Informer { public static String getWindowName() { int MAX_TITLE_LENGTH = 1024; char[] buffer = new char[MAX_TITLE_LENGTH * 2]; HWND hwnd = User32.INSTANCE.GetForegroundWindow(); User32.INSTANCE.GetWindowText(hwnd, buffer, MAX_TITLE_LENGTH); IntByReference pid = new IntByReference(); User32.INSTANCE.GetWindowThreadProcessId(hwnd, pid); HANDLE p = Kernel32.INSTANCE.OpenProcess( Kernel32.PROCESS_QUERY_INFORMATION | Kernel32.PROCESS_VM_READ, false, pid.getValue()); Psapi.INSTANCE.GetModuleBaseNameW(p, null, buffer, MAX_TITLE_LENGTH); return Native.toString(buffer); } public interface Psapi extends StdCallLibrary { @SuppressWarnings("deprecation") Psapi INSTANCE = (Psapi) Native.loadLibrary("Psapi", Psapi.class); WinDef.DWORD GetModuleBaseNameW(HANDLE hProcess, HANDLE hModule, char[] lpBaseName, int nSize); } } for example, the documentation for the GetForegroundWindow function).

With this information it is now possible to wrap the code in a proper source:

public class WindowsWindowMonitorSource implements BeamlineAbstractSource {

   private static final int POLLING_DELAY = 100; // milliseconds between checks of the active window

   @Override
   public void run(SourceContext<BEvent> ctx) throws Exception {
      Queue<BEvent> buffer = new LinkedList<>();

      String caseId = UUID.randomUUID().toString();
      new Thread(new Runnable() {
         @Override
         public void run() {
            String latestProcess = "";
            while(isRunning()) {
               String currentProcess = getWindowName();
               if (!currentProcess.isEmpty() && !currentProcess.equals(latestProcess)) {
                  latestProcess = currentProcess;
                  try {
                     buffer.add(BEvent.create("window", caseId, currentProcess));
                  } catch (EventException e) { }
               }

               try {
                  Thread.sleep(POLLING_DELAY);
               } catch (InterruptedException e) { }
            }
         }
      }).start();

      while(isRunning()) {
         while (isRunning() && buffer.isEmpty()) {
            Thread.sleep(100l);
         }
         if (isRunning()) {
            synchronized (ctx.getCheckpointLock()) {
               BEvent e = buffer.poll();
               ctx.collect(e);
            }
         }
      }
   }
}

The basic idea is to check every POLLING_DELAY milliseconds for the name of the window currently on focus and, if this has changed, then a new event is published.

An example run of the application utilizing the Trivial Miner and the following code:

public class WindowsWindowMonitor {
   public static void main(String[] args) throws Exception {
      StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
      env
         .addSource(new WindowsWindowMonitorSource())
         .keyBy(BEvent::getProcessName)
         .flatMap(new DirectlyFollowsDependencyDiscoveryMiner().setModelRefreshRate(1).setMinDependency(0))
         .addSink(new SinkFunction<ProcessMap>(){
            public void invoke(ProcessMap value, Context context) throws Exception {
               value.generateDot().exportToSvg(new File("src/main/resources/output/output.svg"));
            };
         });
      env.execute();
   }
}

Produces the following map:

The complete code of this example is available in the GitHub repository https://github.com/beamline/examples/tree/master/src/main/java/beamline/examples/windowsWindowMonitor.

It is important to emphasize that the active window might not really be the one that the user is currently using. For example, a user might be reading a webpage in a browser or a PDF document or another text document while having active another window. ↩