Research / Jin Hu
Understanding failures.
Developing reliable AI.
My research advances trustworthy AI through generative evaluation and adaptive optimization. I study how visual and multimodal models respond to changing scenes, semantic constraints, and physical conditions.
A shared question
How can we discover meaningful model failures while controlling the conditions that produce them? My work connects the generation of evaluation examples with the feedback used to optimize them.
Two first-author papers establish complementary foundations: DynamicPAE improves feedback during generator training, while InSUR improves guidance during diffusion inference. Their collaborative extensions explore physical perception and consistent observations across sensing modalities.
Foundational work
First-author publications 01 / Learning from noisy feedback
DynamicPAE
IEEE TPAMI 2026 · First author
Generating Scene-Aware Physical Adversarial Examples in Real-Time
Physical transformations introduce noisy feedback when training an adversarial generator. DynamicPAE learns to generate an example conditioned on the observed scene, making scene adaptation possible through a single forward pass.
- Residual-guided training
- An auxiliary reconstruction task supplies richer feedback and helps the generator explore useful scene-dependent patterns.
- Adaptive objective control
- A controller based on loss-distribution skewness adjusts competing objectives, while conditional scene modeling accounts for incomplete observations and physical variation.
This work establishes a foundation in adaptive optimization for physical reliability evaluation.
02 / Controlling generative inference
InSUR
NeurIPS 2025 · First author
Exploring Semantic-constrained Adversarial Example with Instruction Uncertainty Reduction
Language-guided evaluation requires generated examples to preserve the intended semantics while exposing recognition failures. The InSUR framework connects diffusion guidance, scene constraints, and semantic evaluation.
- Residual approximation for guidance
- ResAdv-DDIM uses a short prediction of the future denoising outcome to stabilize current guidance during diffusion sampling.
- Semantic and scene constraints
- Spatial guidance and differentiable rendering extend generation to 2D images and 3D objects. Abstracted labels and paired examples relate evaluation to the intended semantics.
This work connects controllable visual generation with evaluation of visual and vision-language models.
Collaborative extensions
From foundations to new perception settings InSUR → Multimodal 3D perception
MiuGS
ACM MM 2026 · Accepted · Second author
Modality-Integrated Unified Gaussian Representation for Physical Adversarial Object Generation in 3D Detection
Building on the direction of generative 3D evaluation, MiuGS uses a shared 3D Gaussian representation to produce consistent camera and LiDAR observations of one physical object.
Manuscript not yet public.
Project repository ↗ DynamicPAE → Physical perception
PRCV Oral
PRCV 2026 · Oral accepted · Third author
Dual Bias Suppressed Cross-scale Physical Adversarial Patch against Lane Detection
This collaborative work continues the physical adversarial research line represented by DynamicPAE, with a focus on lane detection.
Manuscript not yet public.
Looking ahead
From failure discovery to model understanding
I am interested in connecting generative evaluation with mechanistic interpretability and the Physics of AI: how learning dynamics and internal computation shape generalization and vulnerability, and how that understanding can guide reliable model behavior.
Complete publication list →