Lindera UniDic CSJ

Dictionary version

This crate builds UniDic for contemporary spoken Japanese (現代話し言葉UniDic) ver.2025.12, from NINJAL's full package unidic-csj-202512_full.zip.

Dictionary format

Refer to the UniDic FAQ for the fields and their values. Fields 0 to 20 are those of UniDic 2.1.2, with the same names, and all 33 are those of UniDic CWJ.

IndexName (Japanese)Name (English)Notes
0表層形Surface
1左文脈IDLeft context ID
2右文脈IDRight context ID
3コストCost
4品詞大分類Part-of-speech
5品詞中分類Part-of-speech subcategory 1
6品詞小分類Part-of-speech subcategory 2
7品詞細分類Part-of-speech subcategory 3
8活用型Conjugation type
9活用形Conjugation form
10語彙素読みReading
11語彙素(語彙素表記 + 語彙素細分類)Lexeme
12書字形出現形Orthographic surface form
13発音形出現形Phonological surface form
14書字形基本形Orthographic base form
15発音形基本形Phonological base form
16語種Word type
17語頭変化型Initial mutation type
18語頭変化形Initial mutation form
19語末変化型Final mutation type
20語末変化形Final mutation form
21語頭変化結合型Initial connection type
22語末変化結合型Final connection type
23語彙素類Lexeme class
24仮名形出現形Kana
25仮名形基本形Kana base
26語形出現形Form
27語形基本形Form base
28アクセント型Accent type
29アクセント結合型Accent connection typeCan contain commas, such as 動詞%F2@0,名詞%F1
30アクセント修飾型Accent modification type
31語彙表IDLexicon ID
32語彙素IDLemma ID

The JSON output of lindera tokenize writes these fields as strings, with the dictionary's text, as it writes every field of the schema. lexicon_id goes above 2^53, the largest integer a JavaScript number holds exactly, so convert it to a number only with a type that keeps big integers, such as JavaScript's BigInt.

User dictionary format (CSV)

Simple version

IndexName (Japanese)Name (English)Notes
0表層形Surface
1品詞大分類Part-of-speech
2語彙素読みReading

Detailed version

The 33 fields of the dictionary format above, in the same order. After index 32, it can be freely expanded.

API reference

The API reference is available. Please see following URL: