[WIP] Issue with ADASYN

[kansuke231]

Hi all,
I am using ADASYN for over-sampling a dataset with multiple classes. Currently what I do is creating a set of pairs of classes from the dataset, and then run ADASYN one by one. Some pairs of classes are fine, but sometimes I see an error as follows:

File "/Users/myname/env/lib/python2.7/site-packages/imblearn/over_sampling/adasyn.py", line 143, in _sample
    nn_zs = random_state.randint(1, high=self.k + 1, size=num_sample_i)
  File "mtrand.pyx", line 1266, in mtrand.RandomState.randint (numpy/random/mtrand/mtrand.c:14775)
  File "mtrand.pyx", line 1267, in mtrand.RandomState.randint (numpy/random/mtrand/mtrand.c:14599)
  File "mtrand.pyx", line 749, in mtrand._rand_int64 (numpy/random/mtrand/mtrand.c:10123)
ValueError: negative dimensions are not allowed

Seems like the problem is coming from

y[ind_nn[:, 1:]] == self.maj_c_

in the line 148 of adasyn.py, as I print out I see the following:

y[ind_nn[:, 1:]] == self.maj_c_  --->
[[False False False]
 [False False False]
 [False False False]
 [False False False]
 [False False False]
 [False False False]
 [False False False]
 [False False False]
 [False False False]
 [False False False]
 [False False False]
 [False False False]
 [False False False]
 [False False False]
 [False False False]
 [False False False]]

Does anyone have any idea why this is occurring? Thank you!

[glemaitre]

@kansuke231 The crash information are coming from line 143 so I don't think it can come from line 148 (by the way I don't understand how you can get information after that line).

Can you give more information regarding X and y that you provide:

print(X.shape)
print(np.unique(y))

Print also the num_sample_i to see if you are generating any sample.

[kansuke231]

@glemaitre Oh sorry for the confusion. I did pip install the other day, but I suppose the installed version is not the one I see currently on Github.

Followings are X and y:

X_extracted.shape: (178, 8)
np.unique(Y_extracted): ['Food Web' 'Fungal']

I inserted a print function right after the line 156 as follows:

        # For each minority samples
        for x_i, x_i_nn, num_sample_i in zip(X_min, ind_nn, num_samples_nn):
            print("num_sample_i",num_sample_i)

And what I get is num_sample_i -9223372036854775808.
Then I suspected num_samples_nn and it actually looked like this:

[-9223372036854775808 -9223372036854775808 -9223372036854775808
 -9223372036854775808 -9223372036854775808 -9223372036854775808
 -9223372036854775808 -9223372036854775808 -9223372036854775808
 -9223372036854775808 -9223372036854775808 -9223372036854775808
 -9223372036854775808 -9223372036854775808 -9223372036854775808
 -9223372036854775808]

which is due to the fact that all entries in normalized ratio_nn is nan (before normalization, everything is 0 and they are divided by 0).

I also inserted print in order to see ind_nn and they looked like this:

[[162 165 166 169]
 [163 170 168 176]
 [164 163 168 166]
 [165 162 173 169]
 [166 168 164 162]
 [167 170 177 169]
 [168 163 176 170]
 [169 168 163 167]
 [170 163 172 174]
 [171 173 169 168]
 [172 174 170 163]
 [173 165 171 169]
 [174 172 170 163]
 [175 176 168 172]
 [176 163 168 170]
 [177 167 169 164]]

As ind_nn looked fine, I thought there was a problem in generating ratio_nn.

[glemaitre]

Can you print the attribute stats_c_.

From the print of ind_nn and y[ind_nn[:, 1:]] == self.maj_c_, it seems that there is no majority samples next to your minority. ADASYN will not be suited in this case and it means that a k-NN will classify your problem without any error.

However just to check: are you features numerical and can you also make your class numerical and check that you have the same issue.

Finally, you can use SMOTE since that it will not be based on the local class distribution around each minority point.

[kansuke231]

stats_c_ is printed as self.stats_c_: Counter({0: 162, 4: 16}).

So I have converted class labels into integer values and still had the same error. I think, as you mentioned, those points in different classes are in fact well-separated, and ADASYN fails to find majority points nearby minorities. Below is how the data set looks like in 3D feature space.

And yes SMOTE works perfectly fine, by the way.

[glemaitre]

Ok so there is not so much that we can apart of adding a meaningful exception for this specific case. This is the limitation of the algorithm itself.

Hope that you are still satisfied :D.