Skip to main content
Open In ColabOpen on GitHub

ModelScope Chat Endpoint

Overview​

ModelScope (Home | GitHub) is built upon the notion of “Model-as-a-Service” (MaaS). It seeks to bring together most advanced machine learning models from the AI community, and streamlines the process of leveraging AI models in real-world applications. The core ModelScope library open-sourced in this repository provides the interfaces and implementations that allow developers to perform model inference, training and evaluation.

This example goes over how to use LangChain to interact with ModelScope Chat Endpoint.

Setup​

Generate your sdk token from: https://modelscope.cn/my/myaccesstoken

Instantiation​

pip install modelscope openai

import os

os.environ["MODELSCOPE_SDK_TOKEN"] = "YOUR_SDK_TOKEN"

Available models: https://modelscope.cn/docs/model-service/API-Inference/intro

Invocation​

from langchain_community.chat_models import ModelScopeChatEndpoint

llm = ModelScopeChatEndpoint(model="Qwen/Qwen2.5-Coder-32B-Instruct")

llm.invoke("write a python program to sort an array")
AIMessage(content='Certainly! Sorting an array can be done in Python using various methods. One of the simplest ways is to use Python\'s built-in sorting functions. However, if you want to implement a sorting algorithm yourself, I\'ll show you how to do it using the Bubble Sort algorithm as an example.\n\nHere\'s a Python program that sorts an array using both the built-in method and the Bubble Sort algorithm:\n\n### Using Built-in Sort\n\n\`\`\`python\ndef sort_array_builtin(arr):\n    return sorted(arr)\n\n# Example usage\narray = [64, 34, 25, 12, 22, 11, 90]\nsorted_array = sort_array_builtin(array)\nprint("Sorted array using built-in method:", sorted_array)\n\`\`\`\n\n### Using Bubble Sort Algorithm\n\n\`\`\`python\ndef bubble_sort(arr):\n    n = len(arr)\n    # Traverse through all array elements\n    for i in range(n):\n        # Last i elements are already in place\n        for j in range(0, n-i-1):\n            # Traverse the array from 0 to n-i-1\n            # Swap if the element found is greater than the next element\n            if arr[j] > arr[j+1]:\n                arr[j], arr[j+1] = arr[j+1], arr[j]\n    return arr\n\n# Example usage\narray = [64, 34, 25, 12, 22, 11, 90]\nsorted_array = bubble_sort(array)\nprint("Sorted array using Bubble Sort:", sorted_array)\n\`\`\`\n\n### Explanation\n\n- **Built-in Sort**: The `sorted()` function is a high-level function that returns a new sorted list from the elements of any iterable. It uses Timsort, which has a time complexity of O(n log n).\n\n- **Bubble Sort**: This is a simple comparison-based algorithm. It repeatedly steps through the list, compares adjacent elements, and swaps them if they are in the wrong order. The pass through the list is repeated until the list is sorted. The time complexity of Bubble Sort is O(n^2), making it inefficient on large lists.\n\nYou can choose either method based on your needs. For most practical purposes, using the built-in `sorted()` function is recommended due to its efficiency and simplicity.', additional_kwargs={}, response_metadata={'token_usage': {'completion_tokens': 476, 'prompt_tokens': 26, 'total_tokens': 502, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'Qwen/Qwen2.5-Coder-32B-Instruct', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-d8369855-b8d3-4def-a39c-8c9bed10900b-0')
# Streaming
for chunk in llm.stream("write a python program to sort an array"):
    print(chunk)

Chaining​

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate(
    [
        (
            "system",
            "You are a helpful assistant that translates {input_language} to {output_language}.",
        ),
        ("human", "{input}"),
    ]
)

chain = prompt | llm
chain.invoke(
    {
        "input_language": "English",
        "output_language": "Chinese",
        "input": "I love programming.",
    }
)
API Reference:ChatPromptTemplate
AIMessage(content='我喜欢编程。', additional_kwargs={}, response_metadata={'token_usage': {'completion_tokens': 3, 'prompt_tokens': 28, 'total_tokens': 31, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'Qwen/Qwen2.5-Coder-32B-Instruct', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-b7602a5e-6008-466d-93d0-c338bbd8c7fb-0')

API reference​

Refer to https://modelscope.cn/docs/model-service/API-Inference/intro for more details.

Was this page helpful?