summaryrefslogtreecommitdiff
path: root/tools/edit_monitor
AgeCommit message (Collapse)Author
2025-02-11Remove long-deprecated assertEquals uses Dan Willemsen
These have been removed in Python 3.12: https://docs.python.org/3/whatsnew/3.12.html#id3 Bug: 388344853 Test: atest edit_monitor_test -- --abi x86_64 Test: atest daemon_manager_test -- --abi x86_64 Change-Id: I8c398390f194ae5e384fe7dab98adbd9d0770f11
2024-12-06Synchronize the setup process of edit monitor Zhuoyao Zhang
This cl ensures only one edit monitor instance is in the setup proecess by requesting exclusive file lock on a setup lock file. If there is already an edit monitor in setup process, other instances trying to setup will fail. This prevents race conditions when multiple edit monitor instance are starting. Test: atest daemon_manager_test bug: 382135550 Change-Id: Ia8bd2f35318ae81fc002ccce62e2fde3b8cae26b
2024-12-06Update the logic to force cleanup all edit monitor instances. Zhuoyao Zhang
Instead of checking the pidfiles, query all the running process with binary ends with "edit_monitor" and kill all of those processes. This ensures that if there are any unexpected edit monitor instances running (e.g. due to b/382135550), the force cleanup can kill those processes as well. Bug: 382135550 Test: atest daemon_manager_test Change-Id: I299f16b7b913e4ee470b5fe9ef0ffdad8d3c0945
2024-12-04Fix a race condition in edit monitor Zhuoyao Zhang
When there are multiple edit monitor starting around the same time, there will be cases that the pidfile is deleted or overridden as multiple processes are trying to remove and create the pidfile around the same time (see details in b/382135550). This cl fix it by checking the pid in the pidfile and only remove the pidfile if the pid stored there is the expected one. Bug: 382135550 Test: atest edit_monitor_integration_test Change-Id: Ifdb4c29c8b36052fea83ce59de1be9d99db3a852
2024-11-21Rollout the edit monitor to 100% user Zhuoyao Zhang
Test: None Bug: 379747487 Change-Id: I02e8061872c9c0666f521c4ea08dcb88338fd87f
2024-11-18Reboot edit monitor when memory exhausted Zhuoyao Zhang
Instead of teminating, reboot the edit monitor when memory usage exceeds the threshold Test: atest daemon_manager_test Bug: 379747487 Change-Id: If90309e720aa0d095bc1eca93521a01ed63575da
2024-11-18Set the edit monitor memory threshold as a percentage Zhuoyao Zhang
Instead of a fixed size of max memory allowed for the edit monitor, set the threshold as a percentage of the total memory. Test: atest daemon_manager_test Bug: 365617369 Change-Id: I5e9ee20ae19f8f918e5a332ebd7ebe4d39ed61d8
2024-11-15Rollout edit monitor to 50% of the users Zhuoyao Zhang
Test: None Bug: 365617369 Change-Id: Idfeb71f63ca8df6bf9aadfb6c69c48b399251647
2024-11-12Increase the memory consumption for edit monitor Zhuoyao Zhang
Test: None Bug: 365617369 Change-Id: Iea82fed5ff5ca2e9ad5c13b30176cb35235eb3f8
2024-11-12Make edit monitor error type clearer Zhuoyao Zhang
Before this change, edit monitor will send a EXCEEDED_RESOURCE_USAGE error to clearcut, however, it does not tell whether it is due to memory or CPU usage overload. This CL separate the errory type to EXCEEDED_MEMORY_USAGE and EXCEEDED_CPU_USAGE to make it easier to debug Test: atest daemon_manager_test Bug: 365617369 Change-Id: Ib0605249d87544c0a2ec82d72f05b2cc9aaa17c1
2024-11-06Change the flag to enable edit monitor to ENABLE_ANDROID_EDIT_MONITOR Zhuoyao Zhang
To prevent conflicts. Test: atest edit_monitor_integartion_test atest daemon_manager_test Bug: 365617369 Change-Id: I6c09c23c2d659d6dc37bc33e00cebf665a7fde0d
2024-11-05Rollout edit monitor to 10% users Zhuoyao Zhang
Simplified the logic in whether to enable edit monitor by passing the rollout percentage directly instead of reading from a env variable as now we do not rely on any env var to control the rollout precentage. Test: atest edit_monitor_utils_test Bug: 365617369 Change-Id: Ia04d5737dafe2c9715a02f84eb5d75ba6119ceb4
2024-11-01Start edit monitor only when the feature is enabled Zhuoyao Zhang
Check the env variable to determine whether the edit monitor feature is enabled and only start the edit monitor if it is enabled, otherwise exit directly. This helps to roll out the feature gradually. Test: atest edit_monitor_utils_test atest daemon_manager_test Bug: 365617369 Change-Id: I03fb494e1f62712efaf0bb05de8859e0118702bf
2024-10-25Support --verbose option in edit monitor Zhuoyao Zhang
If --verbose option is set, logging in DEBUG level, otherwise logging in INFO level, by default, user should run edit monitor without verbose option to reduce the log size. Test: make edit_monitor then run it and check the log file manually Bug: 365617369 Change-Id: Ibdbd2bdd1ff373f7924c60f1b382b14678cf0a92
2024-10-24A few fix of the daemon_manager Zhuoyao Zhang
* Adjust the default process termination timeout to give some more time when waiting process to terminate. * Remove the try...catch block for getting memory and cpu useage as the exception will be caught by the caller anyway. Test: atest daemon_manager_test Bug: 365617369 Change-Id: I91db1cd65045dccc9eaf3cf8ceacbbf5eaf85298
2024-10-24Add integration test for edit monitor Zhuoyao Zhang
Test: atest edit_monitor_integration_test Bug: 365617369 Change-Id: Ie6abe800cf3a7b0755df94748a7e8b02e192a1f4
2024-10-24Update edit_event proto Zhuoyao Zhang
To make it consistent with the proto stored in g3 logs/proto/wireless/android/adte/edit_monitor/edit_event.proto Test: None Change-Id: I93ffa3651563eaf2137cb6b30b00c4d180f84923
2024-10-24Support --dry_run option in edit monitor Zhuoyao Zhang
If --dry_run option is set, edit monitor runs without actually sending the edit logs to clearcut Test: atest edit_monitor_test bug: 365617369 Change-Id: I793d9b0a2e16ffa70fe03d1d3f53d676a1d86cf3
2024-10-24Report edit monitor error to clearcut Zhuoyao Zhang
Report unexpected errors to clearcut when the daemon manager detect anything wrong when start/monitor/stop/reboot the edit monitor process. Test: atest daemon_manager_test Bug: 365617369 Change-Id: I2027229c387ce2525d1f730d06483032662ffc4b
2024-10-23Filter out the logs from inotify_buffer in edit monitor Zhuoyao Zhang
The inotify_buffer logs every filesystem event received (in Debug level) which pollutes the edit monitor log. This cl filters out the logs from inotify_buffer.py to keep the log clean. Test: make edit_monitor and test it locally Bug: 365617369 Change-Id: Ib7951c2fd7d8cabf3f56a2cd0364c26f682ca977
2024-10-23Performance optimization for edit monitor Zhuoyao Zhang
Instead of log every edit event immediately when received the event, cache the events and log the cached events in batch periodically. In case when there are many edits events recieved in a short time (probably due to non-human operation like repo sync), send an aggregated edit event instead to prevent performance degrade. Test: atest edit_monitor_test Bug: 365617369 Change-Id: Ibe1613cf1e2eb37ebc5dfa5c029b990854fcf91e
2024-10-23Ignore the edits unrelated to Android dev in the edit monitor Zhuoyao Zhang
1) Do not log edit events for files that does not belong to a git project. 2) Do not log edit events for hidden files or files under a hidden dir. Test: atest edit_monitor_test bug: 365617369 Change-Id: I73a79d9fb2ec3759ff58bded1666d68cff696353
2024-10-23Add the actual main.py for edit monitor Zhuoyao Zhang
Add the main function as the entry point for the edit monitor. Test: make edit_monitor and test it locally Bug: 365617369 Change-Id: I4d15b45b2ae4c1e90e8408c877754e6db75b46a6
2024-10-21Add the basic edit monitor logic Zhuoyao Zhang
The edit monitor uses the watchdog lib to montior any file changes (create, modify, move, delete) under a specific directory, create a edit log based on the file change event and send the edit log to Sawmill through clearcut. Test: Atest edit_monitor_test Bug: 365617369 Change-Id: Ib6e04a43e4dd78ca58c7ccdef56fb4f6bda6e7a5
2024-10-04Do not start edit monitor under cog workspace Zhuoyao Zhang
Edit monitor for cog workspace is not supported, stop the edit monitor directly if it identifies the cog environment. Test: atest daemon_manager_test Bug: 365617369 Change-Id: I0113983b917490bcc0e5a92d6539626b4dbec6d8
2024-10-02Add a main function for the edit monitor Zhuoyao Zhang
Test: make edit_monitor and run edit_monitor and check the log file bug: 365617369 Change-Id: Iab8e6250e7e11eba6b57932358a4f1cd949c4121
2024-10-02Add the protobuf defintion to store edit event logs Zhuoyao Zhang
Test: make edit_event_proto bug: 365617369 Change-Id: Icffb9a281331cd9f6270487b8a7e74fc94dac008
2024-10-01Support to cleanup all existing edit monitor instances Zhuoyao Zhang
Add a cleanup method in daemon manager to stop all the existing edit monitor instances and place a block sign to provent any edit monitor to start. This method is only used in emergency case when something goes wrong with edit monitor and need immediate cleanup to prevent damage to the system. Test: atest daemon_manager_test bug: 365617369 Change-Id: I8bff4f82a8ce272ccafb4ff8b076f05f56609426
2024-09-30Support restart in daemon manager Zhuoyao Zhang
The daemon manager will restart periodically based on the current binary file. If the edit monitor binary no longer exists (e.g. user has deleted the corresponding Android source tree) when a restart is triggered, the edit monitor will just exit directly. Test: atest daemon_manager_test bug: 365617369 Change-Id: If36ca056c6a45868b5b1f722c07eb2891928e85b
2024-09-25Support monitoring the subprocess in edit monitor Zhuoyao Zhang
The daemon manager will keep monitoring the memory/cpu usage of the daemon process and kill it in case the process is consuming too much resources (specified with a threshold). Test: atest daemon_manager_test bug: 365617369 Change-Id: Ic9f8eb5a338de4e9cf7c8aba381ad752cf6aeba0
2024-09-23Ensure a single running instance of edit monitor Zhuoyao Zhang
This cl ensures at most 1 instance of edit monitor running from the same binary by killing any existing instance before starting. Specifically, When an edit monitor process starts, it will write a pidfile containing its pid and if such pidfile already exists (which means there's another instance there), it will read the pid contained in that pidfile and kill the corresponding process first. Test: atst daemon_manager_test bug: 365617369 Change-Id: I76954344df649aa79a6ef07ce55a62985decdb53
2024-09-19The initial implementation of the edit monitor Zhuoyao Zhang
Creates a class that will manager and monitor the actual edit watchdog that run as a subprocess. As a first step, the class supports: 1) A start method that creates a pidfile and starts a subprocess with given target/args. 2) A stop method that terminates the created subprocess and removes the pidfile. Detailed design in go/android-local-edit-monitor. Test: atest daemon_manager_test bug: 365617369 Change-Id: Ic6d7be67d284ade8033416235b9b0fb1e90e1b1a
2024-09-17Bootstrap the edit monitor project Zhuoyao Zhang
Add basic files required for the edit monitor project. The project aims to monitor file changes on Android source trees that are checked out to local disk and upload the edit event logs for post analysis. More details and design doc can be found in the bug. Bug: b/365617369 Test: None Change-Id: Ie5e2bdb0747823f200c87ebe071f5f9a2e567b46
generated by cgit v1.2.3-59-g8ed1b (git 2.39.0) at 2026-01-07 12:20:59 +0000