ilanguage

A Language Independent 'word finding' tool, useful for stemming, tokenizing, indexing, spell checking and other common NLP tasks. Works on any human language and any unicode character set, learns from the data you give it. (Uses compression, maximum entro

Usage no npm install needed!

<script type="module">
  import ilanguage from 'https://cdn.skypack.dev/ilanguage';
</script>

README

iLanguage

NPM version Build Status Dependency Status

A semi-unsupervised language independent morphological analyzer useful for stemming unknown language text, or getting a rough estimate of possible parses for morphemes in a word. Uses compression, maximum entropy and fieldlinguistics.

Install

$ npm install --save ilanguage

Usage

var iLanguage = require('ilanguage');
var lang = new iLanguage(); 

Lab Members

Post Docs

Interns

Release History

  • v1.0 April 16 2009 - Initial implementation in bash and perl
  • v2.0 Jul 3 2010 - Implementation in C++
  • v3.0 April 30 2011 - Implementation in Groovy
  • v4.0 July 20 2012 - Implementation in JavaScript Map Reduce
  • v4.1 Nov 29 2013 - Added more high level functions for gloss lookup
  • v5.0 Jan 9 2014 - Implementation in CommonJS

License

This project is released under the Apache 2.0 license, which is an very non-restrictive open source license which basically says you can adapt the code to any use you see fit.

How to edit the code

Code style

Sublime will manage this for you if you format (CMD+SHIFT+P, format) your code when you save. You can refer to .editorconfig and .jshintrc for specific options.

Breakpointing while you work

You can open the test/SpecRunner.html in an actual browser to run the unit test file(s) or breakpoint the code.

Modifying the code

In general, you should always ensure that you have the latest Node.js and npm installed. On Mac you can do this by

brew update
brew upgrade node

Test that Gulp is installed by running gulp --version. If the command isn't found, run npm install -g gulp. For more information about installing the tools, see the getting started with Gulp guide.

  1. Fork the repo.
  2. Clone the repo to your computer.
  3. Run npm install to install all build dependencies.
  4. Run gulp to build this project.

Assuming that it looks something like this, you're ready to go. screen shot 2015-06-21 at 5 58 21 pm

Contributing changes

Easy way

  1. Signup for a GitHub account (GitHub is free for OpenSource)
  2. Click on the "Fork" button to create your own copy.
  3. Leave us a note in our issue tracker to tell us a bit about the bug/feature you want to work on.
  4. You can follow the 4 GitHub Help Tutorials to install and use Git on your computer.
  5. Feel free to ask us questions in our issue tracker, we're friendly and welcome Open Source newbies.
  6. Edit the code on your computer, commit it referencing the issue #xx you created ($ git commit -m "fixes #xx i changed blah blah...") and push to your origin ($ git push origin master).
  7. Click on the "Pull Request" button, and leave us a note about what you changed. We will look at your changes and help you bring them into the project!

Advanced way

  1. Create a new branch for new fixes or features, this is easier to build a fix/feature specific pull request than if you work in your master branch directly.
  2. Run gulp watch which will run the tests as you make changes.
  3. Add failing tests for the change you want to make. Run gulp test to see the tests fail.
  4. Fix stuff.
  5. Look at the terminal output (assuming you ran gulp watch) to see if the tests pass. Repeat steps 2-4 until done.
  6. Open test/SpecRunner.html unit test file(s) in actual browser(s) (Chrome Canary, Firefox, Safari) to ensure tests pass everywhere.
  7. Update the documentation to reflect any changes.
  8. Push to your fork and submit a pull request.