{
    "info": {
        "author": "AI21 Labs",
        "author_email": null,
        "bugtrack_url": null,
        "classifiers": [
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.10",
            "Programming Language :: Python :: 3.11",
            "Programming Language :: Python :: 3.12",
            "Programming Language :: Python :: 3.8",
            "Programming Language :: Python :: 3.9"
        ],
        "description": "<h1 align=\"center\">\n    <a href=\"https://github.com/AI21Labs/ai21-tokenizer\">AI21 Labs Tokenizer</a>\n</h1>\n\n<p align=\"center\">\n    <em>A SentencePiece based tokenizer for production uses with AI21's models</em>\n</p>\n\n<p align=\"center\">\n<a href=\"https://github.com/AI21Labs/ai21-tokenizer/actions?query=workflow%3ATest+event%3Apush+branch%3Amain\"><img src=\"https://github.com/AI21Labs/ai21-tokenizer/actions/workflows/test.yaml/badge.svg\" alt=\"Test\"></a>\n<a href=\"https://pypi.org/project/ai21-tokenizer\" target=\"_blank\"><img src=\"https://img.shields.io/pypi/v/ai21-tokenizer?color=%2334D058&label=pypi%20package\" alt=\"Package version\"></a>\n<a href=\"https://pypi.org/project/ai21-tokenizer\" target=\"_blank\"><img src=\"https://img.shields.io/pypi/pyversions/ai21-tokenizer?color=%2334D058\" alt=\"Supported Python versions\"></a>\n<a href=\"https://python-poetry.org/\" target=\"_blank\"><img src=\"https://img.shields.io/endpoint?url=https://python-poetry.org/badge/v0.json\" alt=\"Poetry\"></a>\n<a href=\"https://github.com/semantic-release/semantic-release\" target=\"_blank\"><img src=\"https://img.shields.io/badge/semantic--release-python-e10079?logo=semantic-release\" alt=\"Supported Python versions\"></a>\n<a href=\"https://opensource.org/licenses/Apache-2.0\" target=\"_blank\"><img src=\"https://img.shields.io/badge/License-Apache_2.0-blue.svg\" alt=\"License\"></a>\n</p>\n\n---\n\n## Installation\n\n### pip\n\n```bash\npip install ai21-tokenizer\n```\n\n### poetry\n\n```bash\npoetry add ai21-tokenizer\n```\n\n## Usage\n\n### Tokenizer Creation\n\n### Jamba Tokenizer\n\n```python\nfrom ai21_tokenizer import Tokenizer, PreTrainedTokenizers\n\ntokenizer = Tokenizer.get_tokenizer(PreTrainedTokenizers.JAMBA_INSTRUCT_TOKENIZER)\n# Your code here\n```\n\nAnother way would be to use our Jamba tokenizer directly:\n\n```python\nfrom ai21_tokenizer import JambaInstructTokenizer\n\nmodel_path = \"<Path to your vocabs file>\"\ntokenizer = JambaInstructTokenizer(model_path=model_path)\n# Your code here\n```\n\n#### Async usage\n\n```python\nfrom ai21_tokenizer import Tokenizer, PreTrainedTokenizers\n\ntokenizer = Tokenizer.get_async_tokenizer(PreTrainedTokenizers.JAMBA_INSTRUCT_TOKENIZER)\n# Your code here\n```\n\nAnother way would be to use our async Jamba tokenizer class method create:\n\n```python\nfrom ai21_tokenizer import AsyncJambaInstructTokenizer\n\nmodel_path = \"<Path to your vocabs file>\"\ntokenizer = AsyncJambaInstructTokenizer.create(model_path=model_path)\n# Your code here\n```\n\n### J2 Tokenizer\n\n```python\nfrom ai21_tokenizer import Tokenizer\n\ntokenizer = Tokenizer.get_tokenizer()\n# Your code here\n```\n\nAnother way would be to use our Jurassic model directly:\n\n```python\nfrom ai21_tokenizer import JurassicTokenizer\n\nmodel_path = \"<Path to your vocabs file. This is usually a binary file that end with .model>\"\nconfig = {} # \"dictionary object of your config.json file\"\ntokenizer = JurassicTokenizer(model_path=model_path, config=config)\n```\n\n#### Async usage\n\n```python\nfrom ai21_tokenizer import Tokenizer\n\ntokenizer = Tokenizer.get_async_tokenizer()\n# Your code here\n```\n\nAnother way would be to use our async Jamba tokenizer class method create:\n\n```python\nfrom ai21_tokenizer import AsyncJurassicTokenizer\n\nmodel_path = \"<Path to your vocabs file. This is usually a binary file that end with .model>\"\nconfig = {} # \"dictionary object of your config.json file\"\ntokenizer = AsyncJurassicTokenizer.create(model_path=model_path, config=config)\n# Your code here\n```\n\n### Functions\n\n#### Encode and Decode\n\nThese functions allow you to encode your text to a list of token ids and back to plaintext\n\n```python\ntext_to_encode = \"apple orange banana\"\nencoded_text = tokenizer.encode(text_to_encode)\nprint(f\"Encoded text: {encoded_text}\")\n\ndecoded_text = tokenizer.decode(encoded_text)\nprint(f\"Decoded text: {decoded_text}\")\n```\n\n#### Async\n\n```python\n# Assuming you have created an async tokenizer\ntext_to_encode = \"apple orange banana\"\nencoded_text = await tokenizer.encode(text_to_encode)\nprint(f\"Encoded text: {encoded_text}\")\n\ndecoded_text = await tokenizer.decode(encoded_text)\nprint(f\"Decoded text: {decoded_text}\")\n```\n\n#### What if you had wanted to convert your tokens to ids or vice versa?\n\n```python\ntokens = tokenizer.convert_ids_to_tokens(encoded_text)\nprint(f\"IDs corresponds to Tokens: {tokens}\")\n\nids = tokenizer.convert_tokens_to_ids(tokens)\n```\n\n#### Async\n\n```python\n# Assuming you have created an async tokenizer\ntokens = await tokenizer.convert_ids_to_tokens(encoded_text)\nprint(f\"IDs corresponds to Tokens: {tokens}\")\n\nids = tokenizer.convert_tokens_to_ids(tokens)\n```\n\n**For more examples, please see our [examples](examples) folder.**\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": null,
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "dynamic": null,
        "home_page": null,
        "keywords": null,
        "license": null,
        "maintainer": null,
        "maintainer_email": null,
        "name": "ai21-tokenizer",
        "package_url": "https://pypi.org/project/ai21-tokenizer/",
        "platform": null,
        "project_url": "https://pypi.org/project/ai21-tokenizer/",
        "project_urls": null,
        "provides_extra": null,
        "release_url": "https://pypi.org/project/ai21-tokenizer/0.11.2/",
        "requires_dist": [
            "anyio<5.0.0,>=4.4.0",
            "sentencepiece<1.0.0,>=0.2.0",
            "tokenizers<1.0.0,>=0.15.0"
        ],
        "requires_python": "<4.0,>=3.8",
        "summary": null,
        "version": "0.11.2",
        "yanked": false,
        "yanked_reason": null
    },
    "last_serial": 23763833,
    "releases": {
        "0.10.0": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "8021fd2a31b54b56c8d4cb8dbb62983eaa6f8d5d1e771b047c7d716b66839310",
                    "md5": "839e4391e18406c7b12a08a907f5ce11",
                    "sha256": "a0cf4fed330c5f73c62d6d36adeed2c1da84b27948705536c3417eba82532ead"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.10.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "839e4391e18406c7b12a08a907f5ce11",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": "<4.0,>=3.8",
                "size": 2670646,
                "upload_time": "2024-06-16T17:47:40",
                "upload_time_iso_8601": "2024-06-16T17:47:40.667852Z",
                "url": "https://files.pythonhosted.org/packages/80/21/fd2a31b54b56c8d4cb8dbb62983eaa6f8d5d1e771b047c7d716b66839310/ai21_tokenizer-0.10.0-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "a1eb42c45979d3dd18fb3cf937a0fad57507942318501bf19f3d39a5fba399e7",
                    "md5": "dae68ad29e67165821783a31c583f3f3",
                    "sha256": "e8cae36a830b465edc75529b4e6c2f403d4e89cd4f3fb56c68f393e1aeafc1bf"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.10.0.tar.gz",
                "has_sig": false,
                "md5_digest": "dae68ad29e67165821783a31c583f3f3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": "<4.0,>=3.8",
                "size": 2619737,
                "upload_time": "2024-06-16T17:47:42",
                "upload_time_iso_8601": "2024-06-16T17:47:42.910534Z",
                "url": "https://files.pythonhosted.org/packages/a1/eb/42c45979d3dd18fb3cf937a0fad57507942318501bf19f3d39a5fba399e7/ai21_tokenizer-0.10.0.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.11.0": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "08373050388184222f18abc7198e081ac4661dcc5afe8396fcfa9149385572f7",
                    "md5": "2310eb8c93603417b8230863b7b5afb0",
                    "sha256": "f977eb8b4026a0502c03b47381754502f656f5e48be42967f0fa6fb5c1249a83"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.11.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "2310eb8c93603417b8230863b7b5afb0",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": "<4.0,>=3.8",
                "size": 2673877,
                "upload_time": "2024-06-18T13:00:25",
                "upload_time_iso_8601": "2024-06-18T13:00:25.844727Z",
                "url": "https://files.pythonhosted.org/packages/08/37/3050388184222f18abc7198e081ac4661dcc5afe8396fcfa9149385572f7/ai21_tokenizer-0.11.0-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "32ec8bd8720995a808df7caa88cb50873f59cfeeab960c92a213a9390a32b7cb",
                    "md5": "e7e9eae710d5d8657a925317063c51af",
                    "sha256": "3e592fcf8e64111e13d3ae9e142c662c0a2a57bacf296b48c78dc4a27c60bf26"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.11.0.tar.gz",
                "has_sig": false,
                "md5_digest": "e7e9eae710d5d8657a925317063c51af",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": "<4.0,>=3.8",
                "size": 2621717,
                "upload_time": "2024-06-18T13:00:28",
                "upload_time_iso_8601": "2024-06-18T13:00:28.617529Z",
                "url": "https://files.pythonhosted.org/packages/32/ec/8bd8720995a808df7caa88cb50873f59cfeeab960c92a213a9390a32b7cb/ai21_tokenizer-0.11.0.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.11.1": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "88d44627282833f6fbb9066e32739c9b5016c54eac3f4c8a2aecc371db81fb38",
                    "md5": "d63340cf651fea227244c1b9d42cf9b6",
                    "sha256": "9acc55506ed822fb9c830f88afb3be54fdbc5b9e81607d7dff21eda9129368ce"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.11.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "d63340cf651fea227244c1b9d42cf9b6",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": "<4.0,>=3.8",
                "size": 2673896,
                "upload_time": "2024-06-20T05:47:07",
                "upload_time_iso_8601": "2024-06-20T05:47:07.440578Z",
                "url": "https://files.pythonhosted.org/packages/88/d4/4627282833f6fbb9066e32739c9b5016c54eac3f4c8a2aecc371db81fb38/ai21_tokenizer-0.11.1-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "1d4d8f8e59665af264bd596c8f8907a2494c48ee9bc10b46c9c983422e397e33",
                    "md5": "768561d52a6d1f30ca7231942bf1df51",
                    "sha256": "1776c3354fceda626c5d1bee4e003f63e70bc7d0814bf71198f576b9f5093066"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.11.1.tar.gz",
                "has_sig": false,
                "md5_digest": "768561d52a6d1f30ca7231942bf1df51",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": "<4.0,>=3.8",
                "size": 2621740,
                "upload_time": "2024-06-20T05:47:09",
                "upload_time_iso_8601": "2024-06-20T05:47:09.487478Z",
                "url": "https://files.pythonhosted.org/packages/1d/4d/8f8e59665af264bd596c8f8907a2494c48ee9bc10b46c9c983422e397e33/ai21_tokenizer-0.11.1.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.11.2": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "fa31074c0c65ae16428d38827661b5049ee1bce2e819966d05538df3ada3b8c1",
                    "md5": "fa1d065bf14e7d5c5b7a1624f28c57e8",
                    "sha256": "a9444ca44ef2bffec7cb9f0c3cfa5501dc973cdde0b740e43e137ce9a2f90eab"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.11.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "fa1d065bf14e7d5c5b7a1624f28c57e8",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": "<4.0,>=3.8",
                "size": 2673914,
                "upload_time": "2024-06-20T11:31:22",
                "upload_time_iso_8601": "2024-06-20T11:31:22.701919Z",
                "url": "https://files.pythonhosted.org/packages/fa/31/074c0c65ae16428d38827661b5049ee1bce2e819966d05538df3ada3b8c1/ai21_tokenizer-0.11.2-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "47e95dbbd48dad6867c14abf303f478d2e0e5ffd850dfc903e4b64ea4d0b69fb",
                    "md5": "83bdb85b0cf84df485edd23929cfdf07",
                    "sha256": "35579bca375f071ae6365456f02bd5c9445f408723f7b87646a2bdaa3f57925e"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.11.2.tar.gz",
                "has_sig": false,
                "md5_digest": "83bdb85b0cf84df485edd23929cfdf07",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": "<4.0,>=3.8",
                "size": 2621765,
                "upload_time": "2024-06-20T11:31:24",
                "upload_time_iso_8601": "2024-06-20T11:31:24.496936Z",
                "url": "https://files.pythonhosted.org/packages/47/e9/5dbbd48dad6867c14abf303f478d2e0e5ffd850dfc903e4b64ea4d0b69fb/ai21_tokenizer-0.11.2.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.3.10": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "714ff92808788dd19edcc140a70e1b7cf9bb6bfaf4ec04b99d9a6db64acce758",
                    "md5": "dbc4dd02288582e5b33a844d8693c9f1",
                    "sha256": "6b1c404b7fc75f5526d625cc3462c9ae763dde7a8e7bd17b1e366c1deb5f84ba"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.10-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "dbc4dd02288582e5b33a844d8693c9f1",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2668957,
                "upload_time": "2023-12-27T12:39:51",
                "upload_time_iso_8601": "2023-12-27T12:39:51.352376Z",
                "url": "https://files.pythonhosted.org/packages/71/4f/f92808788dd19edcc140a70e1b7cf9bb6bfaf4ec04b99d9a6db64acce758/ai21_tokenizer-0.3.10-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "43d4e3fbb23b09748313db8ffb810a34e7856183785871a3a3c296a0084fa4ba",
                    "md5": "74ea123e5c998a4c112193f7246616d7",
                    "sha256": "860ec23f9b3b70cd516d010720a6628ea5af64c717f562ceca670d596651e0b7"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.10.tar.gz",
                "has_sig": false,
                "md5_digest": "74ea123e5c998a4c112193f7246616d7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2618891,
                "upload_time": "2023-12-27T12:39:53",
                "upload_time_iso_8601": "2023-12-27T12:39:53.396029Z",
                "url": "https://files.pythonhosted.org/packages/43/d4/e3fbb23b09748313db8ffb810a34e7856183785871a3a3c296a0084fa4ba/ai21_tokenizer-0.3.10.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.3.11": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "65103796cca35f777b04eb4a7f603da50f808068dfe8be6b5e45459c2e2edd27",
                    "md5": "7a720f88ab46f602dbd8bf55c6d9f9c8",
                    "sha256": "80d332c51cab3fa88f0fea7493240a6a5bc38fd24a3d0806d28731d8fc97691f"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.11-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "7a720f88ab46f602dbd8bf55c6d9f9c8",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2668972,
                "upload_time": "2023-12-27T14:35:04",
                "upload_time_iso_8601": "2023-12-27T14:35:04.954226Z",
                "url": "https://files.pythonhosted.org/packages/65/10/3796cca35f777b04eb4a7f603da50f808068dfe8be6b5e45459c2e2edd27/ai21_tokenizer-0.3.11-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "96257ebf855efdc7643e0fc131055a0899318cd069c6d805c04ea8fef96b1259",
                    "md5": "6e88fcb97268661c1e5dcc0a060f3e79",
                    "sha256": "ec11ce4e46d24f71f1c2756ad0de34e0adfd51b5bcd81b544aea13d6935ec905"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.11.tar.gz",
                "has_sig": false,
                "md5_digest": "6e88fcb97268661c1e5dcc0a060f3e79",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2618876,
                "upload_time": "2023-12-27T14:35:07",
                "upload_time_iso_8601": "2023-12-27T14:35:07.475280Z",
                "url": "https://files.pythonhosted.org/packages/96/25/7ebf855efdc7643e0fc131055a0899318cd069c6d805c04ea8fef96b1259/ai21_tokenizer-0.3.11.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.3.5": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "869cafb983569d46623659c2658d824e68a4405ee89d408f906277bae12a24a0",
                    "md5": "1e4637aaf9323602afe782f110c8baa4",
                    "sha256": "4e13174160daef380a017b60eba77e27e7b8df6139d671a3af718117c0eb3bb2"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.5-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1e4637aaf9323602afe782f110c8baa4",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2667427,
                "upload_time": "2023-11-22T13:32:31",
                "upload_time_iso_8601": "2023-11-22T13:32:31.460911Z",
                "url": "https://files.pythonhosted.org/packages/86/9c/afb983569d46623659c2658d824e68a4405ee89d408f906277bae12a24a0/ai21_tokenizer-0.3.5-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "a75ce6310610f305f87987e3e931f1787b56afd15e97075e58bc6ff5a3541be2",
                    "md5": "231397c15914954324bb525c13560aa6",
                    "sha256": "ce5eaa635d9348e7b68d276febec2d763243b42f700d45d1060f61e96230576d"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.5.tar.gz",
                "has_sig": false,
                "md5_digest": "231397c15914954324bb525c13560aa6",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2616864,
                "upload_time": "2023-11-22T13:32:34",
                "upload_time_iso_8601": "2023-11-22T13:32:34.285491Z",
                "url": "https://files.pythonhosted.org/packages/a7/5c/e6310610f305f87987e3e931f1787b56afd15e97075e58bc6ff5a3541be2/ai21_tokenizer-0.3.5.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.3.6": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "1a4fdf4390be61091d935dee19fd6d79f56db945e2254d5b3141c1a1fc7b633d",
                    "md5": "b970240191e023379b047949c08faa7e",
                    "sha256": "1b35fe4968bb9c0a1445071b3872083318352f0a952db70816cd81762e49b26b"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.6-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b970240191e023379b047949c08faa7e",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2667641,
                "upload_time": "2023-11-22T15:27:22",
                "upload_time_iso_8601": "2023-11-22T15:27:22.815598Z",
                "url": "https://files.pythonhosted.org/packages/1a/4f/df4390be61091d935dee19fd6d79f56db945e2254d5b3141c1a1fc7b633d/ai21_tokenizer-0.3.6-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "4c48925147fdafb37772685fef4c3cf2263f41c76182b07ad0a55194f7853c4d",
                    "md5": "814e960ffae05879cd9289407a93d428",
                    "sha256": "1e2599e4a01c4b3e9f985d1d8115e7756d72f233a3e91de0bacc76c2ff2520a6"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.6.tar.gz",
                "has_sig": false,
                "md5_digest": "814e960ffae05879cd9289407a93d428",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2616947,
                "upload_time": "2023-11-22T15:27:27",
                "upload_time_iso_8601": "2023-11-22T15:27:27.870760Z",
                "url": "https://files.pythonhosted.org/packages/4c/48/925147fdafb37772685fef4c3cf2263f41c76182b07ad0a55194f7853c4d/ai21_tokenizer-0.3.6.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.3.7": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "990ca727e8a035b15edd90b1d7e96e10b68b5c1cf88d7a359dc600d0ca82fc3c",
                    "md5": "bc80d2d277d8b74e3847127a2764d885",
                    "sha256": "20befc35e508eafe5e8621dd16ab56540d1b48566dc99d85bd5d8d8150ef640c"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.7-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "bc80d2d277d8b74e3847127a2764d885",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2668215,
                "upload_time": "2023-11-23T11:54:34",
                "upload_time_iso_8601": "2023-11-23T11:54:34.943272Z",
                "url": "https://files.pythonhosted.org/packages/99/0c/a727e8a035b15edd90b1d7e96e10b68b5c1cf88d7a359dc600d0ca82fc3c/ai21_tokenizer-0.3.7-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "0bbf32caf5697ca18db776ce603243c072c9ccf637a3bd45725faea42bef9c57",
                    "md5": "0b9769f0c18d262d84e19bb25ebbad04",
                    "sha256": "16e5db69d40afe6c2991aa297cf66fc878ccddde78c84ebeceffdfb8cad18896"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.7.tar.gz",
                "has_sig": false,
                "md5_digest": "0b9769f0c18d262d84e19bb25ebbad04",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2617773,
                "upload_time": "2023-11-23T11:54:37",
                "upload_time_iso_8601": "2023-11-23T11:54:37.200570Z",
                "url": "https://files.pythonhosted.org/packages/0b/bf/32caf5697ca18db776ce603243c072c9ccf637a3bd45725faea42bef9c57/ai21_tokenizer-0.3.7.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.3.8": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "268635db149a094032cec234739a57cba3b376384dc5be38d0002b5121c61b23",
                    "md5": "7e2ea772e25c60da3297eda0e7cca5be",
                    "sha256": "1bf98e6b364ba577672c3243301e29ed7da6ea539627710ec5e27be4e481bb5b"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.8-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "7e2ea772e25c60da3297eda0e7cca5be",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2668220,
                "upload_time": "2023-11-26T08:08:44",
                "upload_time_iso_8601": "2023-11-26T08:08:44.797542Z",
                "url": "https://files.pythonhosted.org/packages/26/86/35db149a094032cec234739a57cba3b376384dc5be38d0002b5121c61b23/ai21_tokenizer-0.3.8-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "8360409ede2b008b610084f566eacf18c41b01b44e360749f70e4a0e75087a06",
                    "md5": "833ec28f2337e74cd44046dcdc0a50ee",
                    "sha256": "3a9401878b54e13293f5e62b8f23765620bbcba04de1b70840c32b6f2279d9ea"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.8.tar.gz",
                "has_sig": false,
                "md5_digest": "833ec28f2337e74cd44046dcdc0a50ee",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2617790,
                "upload_time": "2023-11-26T08:08:47",
                "upload_time_iso_8601": "2023-11-26T08:08:47.148335Z",
                "url": "https://files.pythonhosted.org/packages/83/60/409ede2b008b610084f566eacf18c41b01b44e360749f70e4a0e75087a06/ai21_tokenizer-0.3.8.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.3.9": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "147124164d593d61bb8ed8a51848ed3160503045d07897baac2e6b9d293877d5",
                    "md5": "3408e9808c91c14b8f32c94357e418fc",
                    "sha256": "10c1d282fee4733378c23aa5512bef02235d6411ed950a3c038767f991f90757"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.9-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "3408e9808c91c14b8f32c94357e418fc",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2668638,
                "upload_time": "2023-11-27T09:08:47",
                "upload_time_iso_8601": "2023-11-27T09:08:47.279339Z",
                "url": "https://files.pythonhosted.org/packages/14/71/24164d593d61bb8ed8a51848ed3160503045d07897baac2e6b9d293877d5/ai21_tokenizer-0.3.9-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "21dede1d00f154fc3c936bbc4e2ccddf959e43bd0760fd76ce2fda7b5132bfd8",
                    "md5": "f94ee0a59ba8e3712ef6f5fd172bf553",
                    "sha256": "e54f433152c3ff15c82cfbfdafbfe616d84401d7a78c0781e7902c9034b44684"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.3.9.tar.gz",
                "has_sig": false,
                "md5_digest": "f94ee0a59ba8e3712ef6f5fd172bf553",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2618591,
                "upload_time": "2023-11-27T09:08:49",
                "upload_time_iso_8601": "2023-11-27T09:08:49.243466Z",
                "url": "https://files.pythonhosted.org/packages/21/de/de1d00f154fc3c936bbc4e2ccddf959e43bd0760fd76ce2fda7b5132bfd8/ai21_tokenizer-0.3.9.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.4.0": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "5134bf1781579ca6e48fe5c1dab2ea7a89f2db2bda342243a5f1154b92e38fdf",
                    "md5": "846e8c3eb14c8759a93180e37a77393a",
                    "sha256": "bfd8c0535f40e5111897a26fe2a207b88c0317a6061b66c776f2f3e852445407"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.4.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "846e8c3eb14c8759a93180e37a77393a",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2668974,
                "upload_time": "2023-12-28T07:46:20",
                "upload_time_iso_8601": "2023-12-28T07:46:20.106311Z",
                "url": "https://files.pythonhosted.org/packages/51/34/bf1781579ca6e48fe5c1dab2ea7a89f2db2bda342243a5f1154b92e38fdf/ai21_tokenizer-0.4.0-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "5bc9c9b888ec4968993d4c5064d323cf1da631668f2597a82fbefb81928ecf8f",
                    "md5": "cfb1d5efbc4fb8b085e6f98cb808f66f",
                    "sha256": "8f4203afd826b8c0edbebb117cd0ed17f7cb64900fe1157c4c6b02d5da24986f"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.4.0.tar.gz",
                "has_sig": false,
                "md5_digest": "cfb1d5efbc4fb8b085e6f98cb808f66f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2618878,
                "upload_time": "2023-12-28T07:46:22",
                "upload_time_iso_8601": "2023-12-28T07:46:22.665052Z",
                "url": "https://files.pythonhosted.org/packages/5b/c9/c9b888ec4968993d4c5064d323cf1da631668f2597a82fbefb81928ecf8f/ai21_tokenizer-0.4.0.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.5.0": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "08ff8c50400e995dcc2226a972e574192438f1016ed853812bf85554b4c8df95",
                    "md5": "542f333fb57a9c0154fe5ae31db27728",
                    "sha256": "dc893a0c80f8a93997d1d73fcf93435125d26fd8b049978e90c1faf388fa584d"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.5.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "542f333fb57a9c0154fe5ae31db27728",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2668979,
                "upload_time": "2023-12-28T11:31:46",
                "upload_time_iso_8601": "2023-12-28T11:31:46.390657Z",
                "url": "https://files.pythonhosted.org/packages/08/ff/8c50400e995dcc2226a972e574192438f1016ed853812bf85554b4c8df95/ai21_tokenizer-0.5.0-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "559b9e2e9ef64dc521327f067a1c4a574a5637eba9fe3b1675e1ec44b90d17c5",
                    "md5": "4ee99f5a280a46369641bcf4775ee9e9",
                    "sha256": "2532cb3e185a5e232d71b9d6bb30ce189ca8e30deed17028736a4162ce1739d7"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.5.0.tar.gz",
                "has_sig": false,
                "md5_digest": "4ee99f5a280a46369641bcf4775ee9e9",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2618909,
                "upload_time": "2023-12-28T11:31:48",
                "upload_time_iso_8601": "2023-12-28T11:31:48.885402Z",
                "url": "https://files.pythonhosted.org/packages/55/9b/9e2e9ef64dc521327f067a1c4a574a5637eba9fe3b1675e1ec44b90d17c5/ai21_tokenizer-0.5.0.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.6.0": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "a3bf5a6dad03395b7134bb73ee62f1294a867481e849a26466755281dca3520f",
                    "md5": "f71140b174bdd5f599113e28b2d54840",
                    "sha256": "987436237d27cb2b123dfbf5b0ad84ee99f2f8fbb35aceb3682ed58ed2c6b781"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.6.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f71140b174bdd5f599113e28b2d54840",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2669081,
                "upload_time": "2023-12-28T14:28:43",
                "upload_time_iso_8601": "2023-12-28T14:28:43.021231Z",
                "url": "https://files.pythonhosted.org/packages/a3/bf/5a6dad03395b7134bb73ee62f1294a867481e849a26466755281dca3520f/ai21_tokenizer-0.6.0-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "955e3b75c698142924e3cfd18889b343dda93925eeb4b3f587230016106d8e98",
                    "md5": "9fa9e51bc9ced02d239e68f20fd53686",
                    "sha256": "faa8b6aa6c0b38489ab7fae651e10a5a1481a5e3fae8c1cd136212fea215d786"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.6.0.tar.gz",
                "has_sig": false,
                "md5_digest": "9fa9e51bc9ced02d239e68f20fd53686",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2618985,
                "upload_time": "2023-12-28T14:28:45",
                "upload_time_iso_8601": "2023-12-28T14:28:45.113284Z",
                "url": "https://files.pythonhosted.org/packages/95/5e/3b75c698142924e3cfd18889b343dda93925eeb4b3f587230016106d8e98/ai21_tokenizer-0.6.0.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.7.0": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "90b8a43658517fa23b98d41249e3605a7d404c98954741dee1e21461f893f441",
                    "md5": "c00e39dd51b04dfa855a99caecfb9ee7",
                    "sha256": "149688f9349ba45bbfb44fb93e68756e00669c07bbbbe18685d300201a0be975"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.7.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "c00e39dd51b04dfa855a99caecfb9ee7",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2669305,
                "upload_time": "2024-01-02T15:47:00",
                "upload_time_iso_8601": "2024-01-02T15:47:00.003461Z",
                "url": "https://files.pythonhosted.org/packages/90/b8/a43658517fa23b98d41249e3605a7d404c98954741dee1e21461f893f441/ai21_tokenizer-0.7.0-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "23b9275953106a86cca54e31bfd7884ce6f59ca029e7418c817b2ce73d92d66d",
                    "md5": "addbce47c637fb38719ec2a9844c7ddd",
                    "sha256": "bcd8f3085b81b5c5136637ddc82bd6ee9513b7a0ce6f0989cb172ad931de46bb"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.7.0.tar.gz",
                "has_sig": false,
                "md5_digest": "addbce47c637fb38719ec2a9844c7ddd",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2619177,
                "upload_time": "2024-01-02T15:47:02",
                "upload_time_iso_8601": "2024-01-02T15:47:02.482658Z",
                "url": "https://files.pythonhosted.org/packages/23/b9/275953106a86cca54e31bfd7884ce6f59ca029e7418c817b2ce73d92d66d/ai21_tokenizer-0.7.0.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.8.0": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "314faa5e5568070f23910ec4eb9c34a74d986a739f06bd1ce093cd5244000db0",
                    "md5": "b41969505e4d01dbfa51199e56e4fc3b",
                    "sha256": "ac22ba7c9ab0e181eef95411170f84ccc16ab1d0ccaa4ec108c7fe84404f45d9"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.8.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b41969505e4d01dbfa51199e56e4fc3b",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2669312,
                "upload_time": "2024-01-03T09:26:41",
                "upload_time_iso_8601": "2024-01-03T09:26:41.687715Z",
                "url": "https://files.pythonhosted.org/packages/31/4f/aa5e5568070f23910ec4eb9c34a74d986a739f06bd1ce093cd5244000db0/ai21_tokenizer-0.8.0-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "78c3b4aad29b84959ec472b04ded74f77c3da94065201a189ae0de020a1e2ea3",
                    "md5": "1c616a898ed5b796f93b49d6872e14fb",
                    "sha256": "3956056836aac2247c9b522d1ffd35f7ed175604591f46265a624e1d399ab6af"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.8.0.tar.gz",
                "has_sig": false,
                "md5_digest": "1c616a898ed5b796f93b49d6872e14fb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2619168,
                "upload_time": "2024-01-03T09:26:45",
                "upload_time_iso_8601": "2024-01-03T09:26:45.497936Z",
                "url": "https://files.pythonhosted.org/packages/78/c3/b4aad29b84959ec472b04ded74f77c3da94065201a189ae0de020a1e2ea3/ai21_tokenizer-0.8.0.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.8.1": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "9cb68b00418d7110a1211565d2cd5070a884e245cd7fccaf0b5972b512cd161b",
                    "md5": "9f2ea87cc26da80130d1b40b812583cb",
                    "sha256": "047929ad76552532e7d654ea7af697b3dfa8887f75f90bc285c86ec8ad2e0560"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.8.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "9f2ea87cc26da80130d1b40b812583cb",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2669314,
                "upload_time": "2024-01-07T11:43:08",
                "upload_time_iso_8601": "2024-01-07T11:43:08.621188Z",
                "url": "https://files.pythonhosted.org/packages/9c/b6/8b00418d7110a1211565d2cd5070a884e245cd7fccaf0b5972b512cd161b/ai21_tokenizer-0.8.1-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "20eab1a880fe8a784ee10c630221cea6ec2ec7050f563241888b7c3a05e37c5a",
                    "md5": "8e82e9858849bcf4171e567e81d3ace8",
                    "sha256": "0aa6574d1ff1bb65e4da367692ecb834dbcd81852f1afdbd38e5b06da7396908"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.8.1.tar.gz",
                "has_sig": false,
                "md5_digest": "8e82e9858849bcf4171e567e81d3ace8",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2619161,
                "upload_time": "2024-01-07T11:43:10",
                "upload_time_iso_8601": "2024-01-07T11:43:10.622924Z",
                "url": "https://files.pythonhosted.org/packages/20/ea/b1a880fe8a784ee10c630221cea6ec2ec7050f563241888b7c3a05e37c5a/ai21_tokenizer-0.8.1.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.8.2": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "0a485631ea58b3fc129792a88d02e5f365f7f6f958695188e3d3fb5e415b3429",
                    "md5": "43f9bef2132f5383722e46b43f9f3196",
                    "sha256": "7f005959c53db035cd79d7302f3a0c656bcdfde7c3f07d850eab0894149ca3e4"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.8.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "43f9bef2132f5383722e46b43f9f3196",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 2669321,
                "upload_time": "2024-03-11T07:57:04",
                "upload_time_iso_8601": "2024-03-11T07:57:04.953212Z",
                "url": "https://files.pythonhosted.org/packages/0a/48/5631ea58b3fc129792a88d02e5f365f7f6f958695188e3d3fb5e415b3429/ai21_tokenizer-0.8.2-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "e12ff723f0ff7c90d1e9431c751a0a0284fdb3a9803d6c61ba8b5709c1721118",
                    "md5": "e996c65a8598582d795fe56d5faa8ad5",
                    "sha256": "208dc49bb33eda05ba73d385aaaa00ecdf22fb5cde49492da08c5e20c1ab8f34"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.8.2.tar.gz",
                "has_sig": false,
                "md5_digest": "e996c65a8598582d795fe56d5faa8ad5",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 2619172,
                "upload_time": "2024-03-11T07:57:07",
                "upload_time_iso_8601": "2024-03-11T07:57:07.519730Z",
                "url": "https://files.pythonhosted.org/packages/e1/2f/f723f0ff7c90d1e9431c751a0a0284fdb3a9803d6c61ba8b5709c1721118/ai21_tokenizer-0.8.2.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.9.0": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "ec1c626d728b174004595db07f44592e4ab6bc9686bc073161037110592aed24",
                    "md5": "b10507e6529b5edd554967d6e28591d3",
                    "sha256": "3e4927f0ef98923f53710405fb476b29102bc7c677c65243043c5671cb642d9e"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.9.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b10507e6529b5edd554967d6e28591d3",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": "<4.0,>=3.7",
                "size": 2670646,
                "upload_time": "2024-03-28T16:09:29",
                "upload_time_iso_8601": "2024-03-28T16:09:29.850786Z",
                "url": "https://files.pythonhosted.org/packages/ec/1c/626d728b174004595db07f44592e4ab6bc9686bc073161037110592aed24/ai21_tokenizer-0.9.0-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "33c3664f4f1cf67ce06285be8cd80d6ca29f5650a43ddc5cf926fde56b1bb085",
                    "md5": "21263d4012844af9523de80bc2a0e6b3",
                    "sha256": "724d368eb74564950edfcd6590da7c3b702c4a8f59f60d36dbec5046e6856579"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.9.0.tar.gz",
                "has_sig": false,
                "md5_digest": "21263d4012844af9523de80bc2a0e6b3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": "<4.0,>=3.7",
                "size": 2619736,
                "upload_time": "2024-03-28T16:09:32",
                "upload_time_iso_8601": "2024-03-28T16:09:32.269970Z",
                "url": "https://files.pythonhosted.org/packages/33/c3/664f4f1cf67ce06285be8cd80d6ca29f5650a43ddc5cf926fde56b1bb085/ai21_tokenizer-0.9.0.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.9.1": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "ab26b2d322e3511e392857d0a12ff458857d04968f15f68a098f6b0972f41817",
                    "md5": "879f75857c92d1f600207270a776da64",
                    "sha256": "53e85ddd74b375e7384e5b5c87ff60e63f60e8615c82ce80f27a062b530798ca"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.9.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "879f75857c92d1f600207270a776da64",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": "<4.0,>=3.7",
                "size": 2670637,
                "upload_time": "2024-05-14T18:29:33",
                "upload_time_iso_8601": "2024-05-14T18:29:33.847181Z",
                "url": "https://files.pythonhosted.org/packages/ab/26/b2d322e3511e392857d0a12ff458857d04968f15f68a098f6b0972f41817/ai21_tokenizer-0.9.1-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "4a071c0fcbc5104224935111279abe6b5469aa9d02467bfc7754593fd68e6246",
                    "md5": "4040531f4de95744c2b070c26f82bf48",
                    "sha256": "28c72122ef1eda6ba03c8ae55916de74a7ee9504c759e9bbae44abe821955df9"
                },
                "downloads": -1,
                "filename": "ai21_tokenizer-0.9.1.tar.gz",
                "has_sig": false,
                "md5_digest": "4040531f4de95744c2b070c26f82bf48",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": "<4.0,>=3.7",
                "size": 2619748,
                "upload_time": "2024-05-14T18:29:35",
                "upload_time_iso_8601": "2024-05-14T18:29:35.953849Z",
                "url": "https://files.pythonhosted.org/packages/4a/07/1c0fcbc5104224935111279abe6b5469aa9d02467bfc7754593fd68e6246/ai21_tokenizer-0.9.1.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "fa31074c0c65ae16428d38827661b5049ee1bce2e819966d05538df3ada3b8c1",
                "md5": "fa1d065bf14e7d5c5b7a1624f28c57e8",
                "sha256": "a9444ca44ef2bffec7cb9f0c3cfa5501dc973cdde0b740e43e137ce9a2f90eab"
            },
            "downloads": -1,
            "filename": "ai21_tokenizer-0.11.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "fa1d065bf14e7d5c5b7a1624f28c57e8",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<4.0,>=3.8",
            "size": 2673914,
            "upload_time": "2024-06-20T11:31:22",
            "upload_time_iso_8601": "2024-06-20T11:31:22.701919Z",
            "url": "https://files.pythonhosted.org/packages/fa/31/074c0c65ae16428d38827661b5049ee1bce2e819966d05538df3ada3b8c1/ai21_tokenizer-0.11.2-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "47e95dbbd48dad6867c14abf303f478d2e0e5ffd850dfc903e4b64ea4d0b69fb",
                "md5": "83bdb85b0cf84df485edd23929cfdf07",
                "sha256": "35579bca375f071ae6365456f02bd5c9445f408723f7b87646a2bdaa3f57925e"
            },
            "downloads": -1,
            "filename": "ai21_tokenizer-0.11.2.tar.gz",
            "has_sig": false,
            "md5_digest": "83bdb85b0cf84df485edd23929cfdf07",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<4.0,>=3.8",
            "size": 2621765,
            "upload_time": "2024-06-20T11:31:24",
            "upload_time_iso_8601": "2024-06-20T11:31:24.496936Z",
            "url": "https://files.pythonhosted.org/packages/47/e9/5dbbd48dad6867c14abf303f478d2e0e5ffd850dfc903e4b64ea4d0b69fb/ai21_tokenizer-0.11.2.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "vulnerabilities": []
}