NaN output in 'Hamed and Rao Modified MK Test'

[joaocandre]

I'm running a modified MK test on a couple of short time series:

import pymannkendall as mk

series1 = [0.41625778, 0.40488883, 0.43044564, 0.44369687, 0.46613348, 0.44420775, 0.45091315, 0.48484614, 0.40252088, 0.43944978, 0.51613973, 0.49302274]
series2 = [0.35257984, 0.38692909, 0.39669828, 0.36296244,0.42035612,0.39374964, 0.41100085, 0.43182076, 0.40815853, 0.45394297, 0.41584767, 0.47399517]

print(mk.hamed_rao_modification_test(series1))
print(mk.hamed_rao_modification_test(series2))

And I'm running into a weird result with the second time series, where var_s has a negative value and z and p values become NaN as a result:

Modified_Mann_Kendall_Test_Hamed_Rao_Approach(trend='increasing', h=True, p=0.03352416523500468, z=2.125748992103599, Tau=0.48484848484848486, s=32.0, var_s=212.66666666666666, slope=0.0070362813636363625, intercept=0.40525276250000003)
/home/joao/.local/lib/python3.8/site-packages/pymannkendall/pymannkendall.py:99: RuntimeWarning: invalid value encountered in sqrt
  z = (s - 1)/np.sqrt(var_s)
Modified_Mann_Kendall_Test_Hamed_Rao_Approach(trend='no trend', h=False, p=nan, z=nan, Tau=0.6666666666666666, s=44.0, var_s=-8.737179487179525, slope=0.008305347500000004, intercept=0.36390027875)

[CallMeJDizzy]

I am also running into the same issue. Var_s is negative, while z and p are NaN, as a result.

data = np.array([152.84518, 100.80645, 247.87286, 638.64406, 515.18837,  33.60215,
       136.08871, 333.42453, 357.8629 ,  84.00538,  16.80108, 302.97939,
       757.59468, 428.2044 , 190.83841, 103.60663, 221.77419])

trend, h, pval, z, Tau, s, var_s, slope, intercept = mk.hamed_rao_modification_test(data)

In [22]: trend,h,pval,z,Tau,s,var_s,slope,intercept
Out[22]: 
('no trend',
 False,
 nan,
 nan,
 0.014705882352941176,
 2.0,
 -253.84411764705888,
 1.4569075000000002,
 210.11893)

I have 6 sets of data that have this issue. The issue seems to occur when n_ns is negative, which is traced back to elements of acf_1 being negative.

IPdb [2]: acf_1
array([ 1.        ,  0.24264706, -0.66911765, -0.5       ,  0.31862745,
        0.39460784, -0.21813725, -0.40686275,  0.05637255,  0.37745098,
        0.19607843, -0.16666667, -0.17647059, -0.02696078,  0.05392157,
        0.0245098 ,  0.        ])

IPdb [3]: sni
-2918.6911764705883

Where indices 2 and 3 fail the requirement of the if statement.
I have also noticed the 'modifiedmk' library in R has this same issue, where NaN is returned.

[msoadw]

I had the same issue with the following data, that results to a sni equal to -42150528.1834886 in the function hamed_rao_modification_test(), thus to a negative n_ns and a negative var_s, and thus to the same error sqrt() applied to a negative value.

As a fix, I suggest to apply abs() to sni. It seems to work for me.

data = [ 5079.28015359  4672.30648998  4385.3346906   3993.91351731
  3095.74854944  2691.62726995  1847.44970831  1170.3724502
  1137.75003837  1685.08453186  2299.37731456  2489.00860383
  2180.41583037  1706.55791085  1453.2831123   1523.50131214
  1713.20588091  1771.14713144  1659.12292387  1566.57606633
  1693.75743316  2037.14148675  2389.20363548  2542.86348314
  2491.52336625  2425.90928186  2531.49191731  2788.28382907
  2978.07601332  2905.39665933  2622.60122299  2426.25709951
  2598.30189734  3118.48513438  3630.09755193  3724.60868881
  3307.16719322  2702.65324085  2380.52602082  2531.78564048
  2888.25541852  2965.66344615  2516.99153822  1775.19501002
  1254.80352875  1298.22748905  1793.82154758  2320.33346948
  2554.61234783  2532.07023345  2518.65503591  2661.58997181
  2809.82537195  2700.83306959  2303.46102511  1922.61861351
  1917.82277661  2313.30979306  2725.76956284  2727.06522356
  2297.63757815  1897.78423339  2047.14095391  2804.33445438
  3663.55196101  3988.2669535   3573.39373316  2799.08258539
  2243.80720573  2152.77345901  2278.6926868   2215.19607054
  1846.21746072  1445.21540152  1349.87988126  1585.88578723
  1848.23371181  1848.72668565  1640.37092879  1558.95861425
  1841.22947285  2334.17353718  2619.15663744  2433.72922289
  1952.50259697  1628.48831454  1751.08850736  2169.14007316
  2454.12575102  2326.35487766  1908.11848925  1562.6915469
  1509.81806046  1623.56649165  1606.25891881  1333.24322912
   999.83308761   929.50192794  1261.50505884  1844.56515058
  2424.26880181  2899.62654478  3373.20402779  3965.32721127
  4622.10795662  5137.91232837  5360.49446571  5341.31747253
  5261.22766519  5219.22941375  5127.51370333  4836.23118539
  4351.91322609  3902.97208669  3766.75847621  4028.82192521
  4518.21210399  4977.32927659  5285.13262325  5513.83082656
  5790.95282182  6141.89552299  6483.92096847  6751.63137476
  6982.53989875  7250.80693037  7531.67259328  7676.23546587
  7555.392352    7225.75102197  6918.84983016  6832.1761131
  6922.21946081  6919.91484645  6571.47595297  5875.27845566
  5084.40692243  4479.49405327  4142.95791727  3949.58972486
  3754.09085764  3556.52475638  3469.93682363  3543.64338634
  3656.525989    3613.42307948  3347.22023328  3004.03483663
  2807.52155132  2834.39292997  2933.29655635  2875.16849377
  2586.15207324  2231.25195253  2070.35125424  2234.34712354
  2636.36761295  3083.27602003  3451.93685729  3749.46285432
  4019.32018092  4222.84910095  4241.97189168  4006.76014655
  3609.28244217  3271.94693581  3182.77864437  3342.42650995
  3560.04777644  3602.91597823  3376.23397097  2994.20988532
  2697.46384458  2684.9289851   2977.16672038  3396.001657
  3669.62668254  3599.19091799  3185.83917724  2636.52781613
  2241.03372418  2200.76953922  2525.31144121  3057.87504847
  3588.00152042  3952.48953548  4067.0095448   3919.10844551
  3582.23141509  3242.58692355  3154.44857091  3475.67678579
  4082.38495972  4562.2963267   4484.41408126  3781.09703268
  2905.4048223   2558.98909484  3157.47684606  4475.712178
  5802.27737936  6495.84369184  6471.61805225  6208.76934841
  6309.65107767  7036.76951788  8224.91194779  9562.35495182
 10882.37125232 12162.51785316 13308.02412146 14061.30588592
 14223.57079875 13964.33465423 13806.85682194 14172.9024105
 14875.33839003 15109.63019452 14074.46424346 11718.90676306
  8912.69677125  6822.10596487  6012.66444921  6073.06653368
  6077.5619805   5430.11505307  4311.89199433  3381.62415079
  3108.09305808  3415.26926575  3924.61990567  4435.91232551
  5078.99124465  5990.62679589  6939.26919756  7401.13317181
  7083.22006823  6342.55567183  5981.83216782  6522.73799021
  7639.56507274  8350.34419065  7852.02956612  6243.92885101
  4460.85407286  3476.42800339  3518.66373811  4002.33573761
  4179.08565648  3839.65310015  3413.51650781  3447.76134674
  4046.64891265  4829.75927695  5389.44090218  5717.93096723
  6146.70306841  6888.19932107  7701.31050116  8067.63547224
  7717.0498479   6957.5481888   6446.47908933  6602.4818029
  7226.44534837  7702.26598342  7583.12548483  6998.38132094
  6499.88460842  6513.75376202  6935.56189061  7251.4894799
  7052.51837708  6438.2235642   5927.48293556  5971.73942037
  6537.61707288  7146.5381452   7322.08994024  7021.21670191
  6655.3213277   6696.40297525  7239.86235151  7917.85055918
  8222.34590107  7927.28650534  7231.97002188  6528.46685521
  6046.65766628  5720.32968376  5373.05207562  4992.91268273
  4780.76432925  4904.93313578  5227.65181738  5336.49186111
  4920.30815538  4147.78542178  3634.14991907  3938.4421829
  5008.73890582  6113.92890607  6399.6429539   5621.40473198
  4396.14702615  3712.11359562  4105.54451786  5226.71691908
  6168.1479487   6234.51153928  5452.93120754  4408.83309975
  3649.86998319  3275.44637805  3068.01285168  2923.69501225
  3041.59867128  3642.35773766  4559.58906997  5242.47371572
  5278.38139471  4928.61520352  5049.27299135  6339.20168583
  8577.18638075 10625.05313202 11298.17380681 10362.57321361
  8745.01686622  7745.52380898  7962.41590263  8890.93658886
  9500.93703225  9186.54251486  8195.30345922  7225.61286769
  6706.22842682  6509.94823764  6311.9243991   6095.46219075
  6189.16283689  6791.00844594  7554.40102986  7786.36892822
  7143.75695449  6115.99707475  5717.75352451  6589.67492174
  8342.41280218  9806.5333735  10004.74980569  8966.77785044
  7649.01151468  7063.14619243  7438.09718061  8154.7726454
  8424.53724452  8004.29101822  7292.90135887  6821.36281987
  6727.13767076  6733.26000547  6568.72444941  6309.76082899
  6254.78407572]```