How to Run Qwen2.5-72B-Instruct Model in the Cloud?

The Qwen2.5-72B-Instruct is the latest large language model from the Qwen Organization. Compared to previous versions of the Qwen Series models, Qwen2.5 is a more robust model that performs exceptionally well in various areas, including coding, mathematics, and instruction-following tasks.

Qwen 2.5 models are released in seven open-source versions: 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B. All models are available under the Apache 2.0 License, except for the 3B and 72B versions, which are available under the Qwen Research and Qwen License.

The Qwen2.5-72B-Instruct model has a context length of up to 128K tokens and can generate text of up to 8K tokens in length. It demonstrates exceptional performance compared to Llama-3.1-405B in various fields, such as coding and mathematics.

Use cases of Qwen2.5-72B-Instruct Model

There are several use cases of the Qwen2.5-72B-Instruct Model:

• Content Generation: Generate long types of content like blogs, articles, stories, reports, documentation, etc.
• Coding Assistance: Supports code generation, debugging, and automation.
• AI Applications: Facilitates the development of chatbots and virtual assistants.

Qwen2.5-72B-Instruct Performance

Performances on Multilingualism

Model Inputs and Outputs

Inputs

• Text prompt describing the questions, instructions, etc.

Outputs

• The primary output of the Qwen2.5-72B-Instruct model is natural language text in long forms, like code, articles, and stories.

In this blog, you'll learn:

- About Qwen2.5-72B-Instruct Model
- Setup GPU-powered Virtual Machine offered by NodeShift
- Run Qwen2.5-72B-Instruct Model in the NodeShift Cloud.

Step-by-Step Process to Run Qwen2.5-72B-Instruct Model in the Cloud

For the purpose of this tutorial, we will use a GPU-powered Virtual Machine offered by NodeShift; however, you can replicate the same steps with any other cloud provider of your choice. NodeShift provides the most affordable Virtual Machines at a scale that meets GDPR, SOC2, and ISO27001 requirements.

Step 1: Sign Up and Set Up a NodeShift Cloud Account

Visit the NodeShift Platform and create an account. Once you've signed up, log into your account.

Follow the account setup process and provide the necessary details and information.

Step 2: Create a GPU Node (Virtual Machine)

GPU Nodes are NodeShift's GPU Virtual Machines, on-demand resources equipped  with diverse GPUs ranging from H100s to A100s. These GPU-powered VMs provide enhanced environmental control, allowing configuration adjustments for GPUs, CPUs, RAM, and Storage based on specific requirements.

Navigate to the menu on the left side. Select the GPU Nodes option, create a GPU Node in the Dashboard, click the Create GPU Node button, and create your first Virtual Machine deployment.

Step 3: Select a Model, Region, and Storage

In the "GPU Nodes" tab, select a GPU Model and Storage according to your needs and the geographical region where you want to launch your model.

We will use 1x H100 SXM GPU for this tutorial to achieve the fastest performance. However, you can choose a more affordable GPU with less VRAM if that better suits your requirements.

Step 4: Select Authentication Method

There are two authentication methods available: Password and SSH Key. SSH keys are a more secure option. To create them, please refer to our official documentation.

Step 5: Choose an Image

Next, you will need to choose an image for your Virtual Machine. We will deploy Qwen2.5-72B-Instruct on an NVIDIA Cuda Virtual Machine. This proprietary, closed-source parallel computing platform will allow you to install Qwen2.5-72B-Instruct on your GPU Node.

After choosing the image, click the 'Create' button, and your Virtual Machine will be deployed.

Step 6: Virtual Machine Successfully Deployed

You will get visual confirmation that your node is up and running.

Step 7: Connect to GPUs using SSH

NodeShift GPUs can be connected to and controlled through a terminal using the SSH key provided during GPU creation.

Once your GPU Node deployment is successfully created and has reached the 'RUNNING' status, you can navigate to the page of your GPU Deployment Instance. Then, click the 'Connect' button in the top right corner.

Now open your terminal and paste the proxy SSH IP or direct SSH IP.

Next, If you want to check the GPU details, run the command below:

nvidia-smi

Step 8: Install Qwen2.5-72B-Instruct

After completing the steps above, it's time to download Qwen2.5 from the Ollama website.

Website Link: https://ollama.com/library/qwen2.5:72b

We will be running Qwen2.5-72B-Instruct. Select the 72B model from the website.

Then run the following command to install Ollama:

curl -fsSL https://ollama.com/install.sh | sh

After the installation process is complete, run the following command to see a list of available commands:

ollama

Next, run the following command to host the Qwen2.5 model so it can be accessed and utilized efficiently.

ollama serve

Step 9: Pull Qwen2.5-72B-Instruct Model

To pull the Qwen2.5-72B-Instruct Model, run the following commands:

ollama pull qwen2.5:72b

Step 10: Run Qwen2.5-72B-Instruct Model

Now, you can run the model in the terminal using the following command and interact with your model:

ollama run qwen2.5:72b

Conclusion

In this guide, we've walked through setting up and running the Qwen2.5-72B-Instruct model Locally on a GPU-powered virtual machine. With Qwen2.5-72B-Instruct, you now have the tools to generate anything like text, code, and maths from prompts. By following these steps, you have the power of advanced AI text generation while enjoying the flexibility and efficiency that Qwen2.5-72B-Instruct offers.

For more information about NodeShift:

• Website
• Docs
• LinkedIn
• X
• Discord
• daily.dev