Machine learning is about optimization. In learning via optimization, we define an error function (loss function) on the model and try to optimize it, i.e., try to find the optimal parameters of the model by optimization techniques borrowed from optimization literature.

Machine learning is about algorithms. In decision/regression trees, we need to write a recursive-learning algorithm to classify the data arriving at each decision node, and we also need to write a recursive code to generate the decision tree structure.

Machine learning is about statistics. We usually assume Gaussian noise on the data, we assume multivariate normal distribution on class covariance matrices in quadratic discriminant analysis, and we use cross-validation or bootstrapping to generate multiple training sets.

Machine learning is about models. In decision trees, the data structure is a binary/L-ary tree depending on the type of the features and the decision function used.

Machine learning is about performance metrics. In classification, we use accuracy if we want to get a crude estimate on the performance of the classifier. If we need more details on pairwise classes, confusion matrix comes in handy. If the dataset has two classes, more metrics follow: precision, recall, true positive rate, false positive rate, F-measure, etc.

What is machine learning about? Like defining an elephant, we need the combination of all of these topics to define machine learning: we start machine learning by assuming a certain model on the data, use algorithmic and/or optimization and/or statistical techniques to learn the parameters of that model (which is sometimes defined as curve fitting), and use performance metrics to evaluate our model/algorithm/classifier.

The most commonly used representative of nonparametric algorithms is the nearest neighbor. The assumption of nearest neighbor is simple, the world does not change much, i.e., similar things perform similarly. Therefore, we only need to store the dataset itself and make the decision on the test instance based on the similarity of it to the instances in the dataset. In other words, the class label of an instance is strongly influenced by its nearby instances.

If the class distributions are assumed to follow Gaussian density, we obtain our first parametric classifier, namely, quadratic discriminant. The number of parameters, i.e., the model complexity is Kd + Kd (d + 1) / 2, the first part is for class means and the second part for class covariance matrices.

We can assume a single shared covariance matrix for all classes. In this case, simplifying the function reduces to our second classifier, namely, linear discriminant. The model complexity is Kd + d (d + 1) / 2, where the first part is for class means and the second part for shared covariance matrix.

When we assume all off-diagonals of the shared covariance matrix are zero; we get the naive Bayes classifier. The model complexity is Kd + d, where the first part is for class means and the second part for the diagonal of the shared covariance matrix.

We further reduce by taking priors equal and a single covariance value s. In this case, we get the nearest mean classifier and the model complexity is only Kd + 1.

Decision trees have a tree-based structure where each non-leaf node m implements a decision function, f_m(x), and each leaf node corresponds to a class decision. Second, they are one of most interpretable learning algorithms available. When written as a set of IF-THEN rules, the decision tree can be transformed into a human-readable format, which then can be modified and/or validated by human experts in their corresponding domains.

Kernel machines, in other words, support vector machines, are maximum margin methods, where the model is written as a weighted sum of support vectors. Kernel machines are discriminative methods, i.e., they are only interested in the instances across the class boundaries in classification, or instances across the regressor in regression. For obtaining the optimal separating hyperplane, kernel machines try to maximize separability, or margin, and write the problem as a quadratic optimization problem, whose solution gives us support vectors.

Artificial neural networks (ANN) take their inspiration from the brain. The brain consists of billions of neurons and these neurons are interconnected and work in parallel, which makes the brain a powerful computing machine. Each neuron is connected through synapses to thousands of neurons and the firing of a neuron depends on those synapses.

There are three types of neurons (units) in ANN. Each unit except the input unit takes an input and calculates an output. Input units represent a single input feature x_i or the bias = +1. Hidden units calculate an intermediate output from its inputs. They first combine their inputs linearly and then use nonlinear activation functions to map that linear combination to a nonlinear space. Output units calculate the output of the ANN.

You can also see Python, Java, C++, C, Swift, Js, or C# repository.

• Python 3.7 or higher
• Git

To check if you have a compatible version of Python installed, use the following command:

python -V

You can find the latest version of Python here.

Install the latest version of Git.

pip3 install NlpToolkit-Classification-Cy

In order to work on code, create a fork from GitHub page. Use Git for cloning the code to your local or below line for Ubuntu:

git clone <your-fork-git-link>

A directory called Classification will be created. Or you can use below link for exploring the code:

git clone https://github.com/starlangsoftware/Classification-Cy.git

Steps for opening the cloned project:

• Start IDE
• Select File | Open from main menu
• Choose Classification-Cy file
• Select open as project option
• Couple of seconds, dependencies will be downloaded.

• Classification Algorithms
• Sampling Strategies
• Feature Selection
• Statistical Tests

Algoritmaları eğitmek için

train(self, trainSet: InstanceList, parameters: Parameter)

Eğitilen modeli bir veri örneği üstünde sınamak için

predict(self, instance: Instance) -> str

Karar ağacı algoritması C45 sınıfında

Bagging algoritması Bagging sınıfında

Derin öğrenme algoritması DeepNetwork sınıfında

KMeans algoritması KMeans sınıfında

Doğrusal ve doğrusal olmayan çok katmanlı perceptron LinearPerceptron ve MultiLayerPerceptron sınıflarında

Naive Bayes algoritması NaiveBayes sınıfında

K en yakın komşu algoritması Knn sınıfında

Doğrusal kesme analizi algoritması Lda sınıfında

İkinci derece kesme analizi algoritması Qda sınıfında

Destek vektör makineleri algoritması Svm sınıfında

RandomForest ağaç tabanlı ensemble algoritması RandomForest sınıfında

Basit dummy ve rasgele sınıflandırıcı gibi temel iki sınıflandırıcı Dummy ve RandomClassifier sınıflarında

K katlı çapraz geçerleme deneyi yapmak için KFoldRun, KFoldRunSeparateTest, StratifiedKFoldRun, StratifiedKFoldRunSeparateTest

M tane K katlı çapraz geçerleme deneyi yapmak için MxKFoldRun, MxKFoldRunSeparateTest, StratifiedMxKFoldRun, StratifiedMxKFoldRunSeparateTest

Bootstrap tipi deney yapmak için BootstrapRun

Pca tabanlı boyut azaltma için Pca sınıfı

Discrete değişkenleri Continuous değişkenlere çevirmek için DiscreteToContinuous sınıfı

Discrete değişkenleri binary değişkenlere değiştirmek için LaryToBinary sınıfı

İstatistiksel testler için Combined5x2F, Combined5x2t, Paired5x2t, Pairedt, Sign sınıfları

1. Do not forget to set package list. All subfolders should be added to the package list.

    packages=['Classification', 'Classification.Model', 'Classification.Model.DecisionTree',
              'Classification.Model.Ensemble', 'Classification.Model.NeuralNetwork',
              'Classification.Model.NonParametric', 'Classification.Model.Parametric',
              'Classification.Filter', 'Classification.DataSet', 'Classification.Instance', 'Classification.Attribute',
              'Classification.Parameter', 'Classification.Experiment',
              'Classification.Performance', 'Classification.InstanceList', 'Classification.DistanceMetric',
              'Classification.StatisticalTest', 'Classification.FeatureSelection'],

2. Package name should be lowercase and only may include _ character.

    name='nlptoolkit_math',

3. Package data should be defined and must ibclude pyx, pxd, c and py files.

    package_data={'NGram': ['*.pxd', '*.pyx', '*.c', '*.py']},

4. Setup should include ext_modules with compiler directives.

    ext_modules=cythonize(["NGram/*.pyx"],
                          compiler_directives={'language_level': "3"}),

1. Define the class variables and class methods in the pxd file.

cdef class DiscreteDistribution(dict):

    cdef float __sum

    cpdef addItem(self, str item)
    cpdef removeItem(self, str item)
    cpdef addDistribution(self, DiscreteDistribution distribution)

2. For default values in class method declarations, use *.

    cpdef list constructIdiomLiterals(self, FsmMorphologicalAnalyzer fsm, MorphologicalParse morphologicalParse1,
                               MetamorphicParse metaParse1, MorphologicalParse morphologicalParse2,
                               MetamorphicParse metaParse2, MorphologicalParse morphologicalParse3 = *,
                               MetamorphicParse metaParse3 = *)

3. Define the class name as cdef, class methods as cpdef, and __init__ as def.

cdef class DiscreteDistribution(dict):

    def __init__(self, **kwargs):
        """
        A constructor of DiscreteDistribution class which calls its super class.
        """
        super().__init__(**kwargs)
        self.__sum = 0.0

    cpdef addItem(self, str item):

4. Do not forget to comment each function.

    cpdef addItem(self, str item):
        """
        The addItem method takes a String item as an input and if this map contains a mapping for the item it puts the
        item with given value + 1, else it puts item with value of 1.

        PARAMETERS
        ----------
        item : string
            String input.
        """

5. Function names should follow caml case.

    cpdef addItem(self, str item):

6. Local variables should follow snake case.

	det = 1.0
	copy_of_matrix = copy.deepcopy(self)

7. Variable types should be defined for function parameters, class variables.

    cpdef double getValue(self, int rowNo, int colNo):

8. Local variables should be defined with types.

    cpdef sortDefinitions(self):
        cdef int i, j
        cdef str tmp

9. For abstract methods, use ABC package and declare them with @abstractmethod.

    @abstractmethod
    def train(self, train_set: list[Tensor]):
        pass

10. For private methods, use __ as prefix in their names.

    cpdef list __linearRegressionOnCountsOfCounts(self, list countsOfCounts)

11. For private class variables, use __ as prefix in their names.

cdef class NGram:
    cdef int __N
    cdef double __lambda1, __lambda2
    cdef bint __interpolated
    cdef set __vocabulary
    cdef list __probability_of_unseen

12. Write __repr__ class methods as toString methods
13. Write getter and setter class methods.

    cpdef int getN(self)
    cpdef setN(self, int N)

14. If there are multiple constructors for a class, define them as constructor1, constructor2, ..., then from the original constructor call these methods.

cdef class NGram:

    cpdef constructor1(self, int N, list corpus):
    cpdef constructor2(self, str fileName):
    def __init__(self,
                 NorFileName,
                 corpus=None):
        if isinstance(NorFileName, int):
            self.constructor1(NorFileName, corpus)
        else:
            self.constructor2(NorFileName)

15. Extend test classes from unittest and use separate unit test methods.

class NGramTest(unittest.TestCase):

    def test_GetCountSimple(self):

16. For undefined types use object as type in the type declarations.

cdef class WordNet:

    cdef object __syn_set_list
    cdef object __literal_list

17. For boolean types use bint as type in the type declarations.

	cdef bint is_done

18. Enumerated types should be used when necessary as enum classes, and should be declared in py files.

class AttributeType(Enum):
    """
    Continuous Attribute
    """
    CONTINUOUS = auto()
    """

19. Resource files should be taken from pkg_recources package.

	fileName = pkg_resources.resource_filename(__name__, 'data/turkish_wordnet.xml')