System and Method for Closed-Loop Uncertainty for Human-Machine Teamwork
US20250036982A1
Abstract
A method that includes receiving user input associated with identifying a threshold point associated with a classification task, identifying, a machine learning model, in the first set of visual data, a machine placement candidate point associated with identifying the threshold point, and identifying, based on the machine placement candidate point, a set of baseline confidence values via a baseline uncertainty model. The method includes training the machine learning model based on a determined state space by identifying, subsequent sets of visual data additional threshold points, receiving user feedback indicating an accuracy, comparing the baseline confidence values with locations associated with the additional threshold points, generating reward values based on an identified amount of error, and configuring the machine learning model based on the reward values. The method includes identifying in a second set of visual data, via the trained machine learning model, a visual feature associated with the classification task.
Description (excerpt)
CROSS-REFERENCE This Application is a nonprovisional application of and claims the benefit of priority under 35 U.S.C. § 119 based on U.S. Provisional Patent Application No. 63/528,746 filed Jul. 25, 2023. The Provisional Application and all references cited herein are hereby incorporated by reference into the present disclosure in their entirety. FEDERALLY-SPONSORED RESEARCH AND DEVELOPMENT The United States Government has ownership rights in this invention. Licensing inquiries may be directed to Office of Technology Transfer, US Naval Research Laboratory, Code 1004, Washington, DC 20375, USA; +1.202.767.7230; nrltechtran@us.navy.mil, referencing Navy Case #211622. TECHNICAL FIELD The present disclosure is related to machine learning, and more specifically to closed-loop uncertainty and the probability of machine correctness. BACKGROUND Calibrating uncertainty in online machine learning can be done by training an isotonic regression (Kuleshov, Volodymyr, Nathan Fenner, and Stefano Ermon. “Accurate uncertainties for deep learning using calibrated regression.” International conference on machine learning. PMLR, 2018) or logistic regression (Kuleshov, Volodymyr, Nathan Fenner, and Stefano Ermon. “Accurate uncertainties for deep learning using calibrated regression.” International conference on machine learning. PMLR, 2018.) model to learn the relationship between predicted confidence values and the observed accuracy of the classifier. Concept drift can be detected using automated methods like measuring entropy (Kuleshov, Volodymyr, Nathan Fenner, and Stefano Ermon. “Accurate uncertainties for deep learning using calibrated regression.” International conference on machine learning. PMLR, 2018). This method can lead to poor uncertainty quantification until after the concept drift is detected, and calibration can occur. Other work regarding employing reinforcement learning for calibration has relied on the usage of neural networks (Tian, Yuan, et al. “Real-time model calibration with deep reinforcement learning.” Mechanical Systems and Signal Processing 165 (2022): 108284). These methods typically need a large amount of training data, and can be less resilient in an online setting. Human-in-the-loop reinforcement learning has been employed to achieve tasks such as automated driving (Liang, Huanghuang, et al. “Human-in-the-loop reinforcement learning.” 2017 Chinese Automation Congress (CAC). IEEE, 2017). However, these methods have often insufficiently addressed the aspect of uncertainty. SUMMARY This summary is intended to introduce, in simplified form, a selection of concepts that are further described in the Detailed Description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. Instead, it is merely presented as a brief overview of the subject matter described and claimed herein. The present disclosure provides for a method that includes providing, by a processing device, a first set of visual data, receiving, by the processing device, user input associated with identifying a threshold point in the first set of visual data, the threshold point being associated with a classification task, and identifying, by the processing device via a machine learning model, in the first set of visual data, a machine placement candidate point associated with identifying the threshold point. The method may include identifying, based on the machine placement candidate point, a set of baseline confidence values via a baseline uncertainty model, and determining, based on the set of baseline confidence values, a state space, wherein the determining comprises determining differences between successive baseline confidences values in the set of baseline confidence values. The method may include training, by the processing device, the machine learning model based on the determined state space, wherein training comprises (i) identifying, via the machine learning model, in one or more subsequent sets of visual data additional threshold points associated with the classification task, (ii) receiving user feedback indicating an accuracy associated with each of the additional threshold points, (iii) comparing the baseline confidence values with locations associated with the additional threshold points, (iv) identifying an amount of error associated with a window of the additional threshold points, (v) generating reward values based on the identified amount of error, and (vi) configuring the machine learning model based on the generated reward values. The method may include identifying, by the processing device, via the trained machine learning model, a visual feature in a second set of visual data, the visual feature being
Filing details
- Inventors
- Zachary A. Bishof
- Assignee
- The Government Of The United States Of America, As Represented By The Secretary …
- Filed
- Jul 25, 2024
- Granted
- Application pending
Bibliographic data and excerpted text sourced from Google Patents (public record) as part of IP TechMatch's current-filings monitor. This filing is not part of the 2019 historical archive. For the authoritative full text, drawings, and legal status, see the source links above or consult USPTO records directly.