Troubleshooting the Endpoint Agent

Introduction

When an endpoint agent is offline or not properly installed, it is not able to send endpoint data to the MDR appliance leaving a monitoring blind spot. An offline agent is also not able to receive EDR tasking from Field Effect security analysts, further impairing the security of the endpoint.

Agent performance issues may be reported by users, or may be visible in performance monitoring data. An agent that is impacting performance on a host may be due to the EDR component or the user-mode monitoring component, and Field Effect has troubleshooting tools to investigate these issues.

This article walks through some common troubleshooting procedures for the endpoint agent.

Issues Installing the Endpoint Agent

Windows

Check the following common troubleshooting procedures:

Confirm whether the installation is manual, GPO or RMM.
Ensure execution is not run from inside the installer zip file.
Ensure the license.txt file is stored in the same folder as the MSI file.
Check if the command line installation works and collect the msi_install_log.txt log file to verify permissions, paths and license key
```
msiexec.exe /i agent_installer_filename.msi COVALENCE_LICENSE="license-from-txt-file" /qn /l*v msi_install_log.txt 
```
Check the operating system and processor/architecture. For x64 systems, the 64-bit installer must be run; the 32-bit installer must be used on 32-bit OS. See Endpoint Agent Requirements for more.
There may be interfering installed components from earlier installs, so manually uninstall and reinstall the agent.

Linux

Try installing the agent using verbose logging using the two rpm commands below:

$ rpm -qp --scripts covalence-endpoint-x…y…z.rpm 

$ rpm -ivv covalence-endpoint-x…y…z.rpm

Not seeing installed agents on the Appliance or MDR Portal

Check the following common troubleshooting procedures:

Ensure you are using the most recent agent installer, since crypto certs expire regularly, and installing an agent with an invalid cert will not report into the physical appliance.
Check whether the agent shows up on the appliance/portal after ~30 minutes, as the new agent is not visible immediately after installing.
Ensure the agent has network access via your firewall to following the domains (you should be able to successfully ping to epid.fieldeffect.net and installlogs.fieldeffect.net
- epid.fieldeffect.net, TCP/443
- installlogs.fieldeffect.net, TCP/443
Collect status.json file to confirm version and whether the agent is checking in properly. In the status.json file, the StateName must be ACTIVE. Any other state indicates that the agent is not reaching the identify server or the appliance. Json files can be found at the following locations:
- Windows - C:\ProgramData\Field Effect\Covalence\data
- Linux - /opt/fieldeffect/covalence-endpoint/data
- Mac - /Library/Application Support/Covalence/data/status.json

Performance Issues

Windows

Allow Field Effect if you use Multiple Security Solutions

To eliminate the possibility of an interaction with a competing EDR, please be sure that Field Effect has been added to the exclusion list of any other security application you may have in place. The following directories should be added:

```
C:\Program Files\Field Effect
```
```
C:\ProgramData\Field Effect
```

C:\Windows\system32\drivers\CovAgent.sys

Allow for Patching Updates

Windows' routine patching cycles may push the agent into overload while it scans those update processes, So, please confirm that the overload is not related to Patch Tuesday.

macOS

Ensure Full Disk Access is enabled

Begin by verifying that the system extension is installed, and that FDA (Full Disk Access) is enabled for the installation.

For Full Disk Access to be set, the system relies on the system extension to be allowed first. This can be can be set as part of an MDM deployment by using the sysext and tcc mobilconfig files found within the MacOS installer zip file. By using these config files in the MDM, the user is not prompted to accept any permissions, eliminating the risk that Field Effect MDR is denied any required permissions.

FDA cannot be remotely checked, but it can be checked locally on the device, though not always from the Apple icon > System Preferences > Security & Privacy pane, as it only shows permissions that are allowed by allowed by end users via a prompt. Settings pushed via MDM profiles are not listed with the command. It should be noted that some RMM/MDM tools have their own features to check on FDA settings.

To verify the MDM installation, check the Profiles pane in System Preferences for the payloads pushed down. You can also look within the MDMOverrides.plist file with the following sudo command in a Terminal window (Terminal must itself have full disk access permission set to access this file):

/usr/libexec/PlistBuddy -c "print" /Library/Application Support/com.apple.TCC/MDMOverrides.plist

To narrow down the problem and take out uncertainty around MDM settings, uninstall a single agent from one Mac device and manually install the agent according to the Installing the Agent on macOS support guide, while accepting the system ext. and FDA prompts.

Verify the system extension exists

Check that covalence-esext is showing up under “All Processes” in Activity Monitor. A good working status will show [Activation Enabled], while a bad status would be [activated waiting for user], [waiting for approval], or [blocked].

Linux

Allow Field Effect if you use Multiple Security Solutions

If you have another security product in use, add Field Effect MDR to the exceptions:

Programs:

/opt/fieldeffect/covalence-endpoint/bin/covalence-endpoint
/opt/fieldeffect/covalence-endpoint/bin/covalence-health-service

Directories:

/opt/fieldeffect/covalence-endpoint

Files:

/opt/fieldeffect/covalence-endpoint/bin/covalence.ko