Instructions to use MathLLMs/FigCodifier with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MathLLMs/FigCodifier with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="MathLLMs/FigCodifier", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("MathLLMs/FigCodifier", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use MathLLMs/FigCodifier with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MathLLMs/FigCodifier"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MathLLMs/FigCodifier",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/MathLLMs/FigCodifier

SGLang

How to use MathLLMs/FigCodifier with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MathLLMs/FigCodifier" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MathLLMs/FigCodifier",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MathLLMs/FigCodifier" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MathLLMs/FigCodifier",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use MathLLMs/FigCodifier with Docker Model Runner:
```
docker model run hf.co/MathLLMs/FigCodifier
```

FigCodifier / README.md

scikkk

Update README.md

9e84660 verified 7 months ago

preview code

raw

history blame contribute delete

3.79 kB

	---
	license: apache-2.0
	language:
	- en
	metrics:
	- accuracy
	pipeline_tag: image-text-to-text
	tags:
	- mathematics
	- reasoning
	- multi-modal-qa
	- math-qa
	- figure-qa
	- geometry-qa
	- math-word-problem
	- textbook-qa
	- vqa
	- geometry-diagram
	- synthetic-scene
	- chart
	- plot
	- scientific-figure
	- table
	- function-plot
	- abstract-scene
	- puzzle-test
	- document-image
	- science
	library_name: transformers
	base_model:
	- OpenGVLab/InternVL2-8B
	datasets:
	- MathLLMs/ImgCode-8.6M
	---
	# MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning

	Repo: [https://github.com/mathllm/MathCoder](https://github.com/mathllm/MathCoder)

	Paper: [https://huggingface.co/papers/2505.10557](https://huggingface.co/papers/2505.10557)


	## Introduction
	We introduce MathCoder-VL, a series of open-source large multimodal models (LMMs) specifically tailored for general math problem-solving. We also introduce [FigCodifier-8B](https://huggingface.co/MathLLMs/FigCodifier), an image-to-code model trained with [ImgCode-8.6M](https://huggingface.co/datasets/MathLLMs/ImgCode-8.6M).

	\| Base Model \|Ours \|
	\|-------------------------------------------------------------------\|-----------------------------------------------------------------------\|
	\| [Mini-InternVL-Chat-2B-V1-5](https://huggingface.co/OpenGVLab/Mini-InternVL-Chat-2B-V1-5) \| [MathCoder-VL-2B](https://huggingface.co/MathLLMs/MathCoder-VL-2B) \|
	\| [InternVL2-8B](https://huggingface.co/OpenGVLab/InternVL2-8B) \| [MathCoder-VL-8B](https://huggingface.co/MathLLMs/MathCoder-VL-8B)\|
	\| [InternVL2-8B](https://huggingface.co/OpenGVLab/InternVL2-8B) \| [FigCodifier-8B](https://huggingface.co/MathLLMs/FigCodifier)\|



	## Usage
	For training and inference code, please refer to [InternVL2-8B](https://huggingface.co/OpenGVLab/InternVL2-8B).



	### Prompt for TikZ Code Generation

	```
	<image>\nPlease generate the corresponding TikZ code that accurately represents the visual elements in the image. TikZ is a powerful tool for creating vector graphics within LaTeX documents. Your generated code should be precise, well-structured, and should recreate the image as faithfully as possible.
	```

	<div align="center">
	<img src="./examples/tikzimage.png" width="100%" title="Result Figure">
	</div>

	### Prompt for Python Code Generation

	```
	Please provide the Python code needed to reproduce this image.\n<image>
	```

	<div align="center">
	<img src="./examples/pyimage.png" width="100%" title="Result Figure">
	</div>


	## Motivation

	<div align="center">
	<img src="./examples/fig1.png" width="100%" title="Result Figure">
	</div>

	## Construction of FigCodifier

	<div align="center">
	<img src="./examples/fig2.png" width="100%" title="Result Figure">
	</div>



	## Citation

	Please cite the paper if you use our data, model or code.

	```
	@inproceedings{
	wang2025mathcodervl,
	title={MathCoder-{VL}: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning},
	author={Ke Wang and Junting Pan and Linda Wei and Aojun Zhou and Weikang Shi and Zimu Lu and Han Xiao and Yunqiao Yang and Houxing Ren and Mingjie Zhan and Hongsheng Li},
	booktitle={The 63rd Annual Meeting of the Association for Computational Linguistics},
	year={2025},
	url={https://openreview.net/forum?id=nuvtX1imAb}
	}

	@inproceedings{
	wang2024mathcoder,
	title={MathCoder: Seamless Code Integration in {LLM}s for Enhanced Mathematical Reasoning},
	author={Ke Wang and Houxing Ren and Aojun Zhou and Zimu Lu and Sichun Luo and Weikang Shi and Renrui Zhang and Linqi Song and Mingjie Zhan and Hongsheng Li},
	booktitle={The Twelfth International Conference on Learning Representations},
	year={2024},
	url={https://openreview.net/forum?id=z8TW0ttBPp}
	}
	```