Parameter efficient multi-model vision assistant for polymer solvation behaviour inference

npj

Polymer–solvent systems exhibit complex solvation behaviours encompassing a diverse range of phenomena, including swelling, gelation, and dispersion. Accurate interpretation is often hindered by subjectivity, particularly in manual rapid screening assessments. While computer vision models hold significant promise to replace the reliance on human evaluation for inference, their adoption is limited by the lack of domain-specific datasets tailored, in our case, to polymer–solvent systems. To bridge this gap, we conducted extensive screenings of polymers with diverse physical and chemical properties across various solvents, capturing solvation characteristics through images, videos, and image–text captions. This dataset informed the development of a multi-model vision assistant, integrating computer vision and vision-language approaches to autonomously detect, infer, and contextualise polymer–solvent interactions. The system combines a 2D-CNN module for static solvation state classification, a hybrid 2D/3D-CNNmodule to capture temporal dynamics, and a BLIP- 2-based contextualisationmodule to generate descriptive captions for solvation behaviours, including vial orientation, solvent discolouration, and polymer interaction states. Computationally efficient, this vision assistant provides an accurate, objective, and scalable solution in interpreting solvation behaviours, fit for autonomous platforms and high-throughput workflows in material discovery and analysis.

For details: 

Parameter efficient multi-model vision assistant for polymer solvation behaviour inference

Zheng Jie Liew 1,2, Ziad Elkhaiary 1 & Alexei A. Lapkin 1,2

1) Department of Chemical Engineering and Biotechnology, University of Cambridge, Cambridge, CB3 0AS, UK
2) Yusuf Hamied Department of Chemistry, Innovation Centre in Digital Molecular Technologies, University of Cambridge, Cambridge, CB2 1EW, UK

npj
https://www.nature.com/articles/s41524-025-01658-7