微软开源｜纯视觉GUI屏幕解析工具：OmniParser
OmniParser是一种将用户界面截图解析为结构化和易于理解的元素的综合工具，它显著增强了GPT-4V生成可以在界面相应区域准确定位的动作的能力。安装步骤：cd OmniParserconda create -n "omni" python==3.12
conda activate omni
pip install -r requirements.txt
然后确保在weights文件夹中有V2权重（确保标题权重文件夹名为icon_caption_florence）。可以通过如下命令实现：# download the model checkpoints to local directory OmniParser/weights/
for f in icon_detect/{train_args.yaml,model.pt,model.yaml} icon_caption/{config.json,generation_config.json,model.safetensors}; do huggingface-cli download microsoft/OmniParser-v2.0 "$f" --local-dir weights; done
mv weights/icon_caption weights/icon_caption_florence
运行：
python gradio_demo.py
项目地址：https://github.com/microsoft/OmniParser
页:
[1]