In This Article, You Will Learn About Percentile & Data Distribution.
Machine Learning Data Distribution – Before moving ahead, let’s take a look at Introduction to Machine Learning SD
Table of Contents
Percentile
What is Percentile?
In simple words, it is a way to show where an observation meets requirements within a range of another observation.
In statistics, Percentile is a number that indicates the value at which a specific percentage of values is less than.
Example: A record contains ten student’s marks.
Marks = [50, 58, 80, 45, 62, 86, 78, 82, 96, 51]
What is meaning of 80 percentiles? It returns 82.8 as answer. Meaning that 80% percent of students got 82.8 or higher marks.
The NumPy module has a method called percentile() method to find out the percentile.
Example: Use percentile() method to find the percentile.
import numpy as np
Marks = [50, 58, 80, 45, 62, 86, 78, 82, 96, 51]
x = np.percentile(Marks, 80)
print (x)
Output -
82.8
As a result, it returned a value that shows how many marks students got above 80%.
Example: What is the number that 45% of students got higher marks.
import numpy as np
Marks = [50, 58, 80, 45, 62, 86, 78, 82, 96, 51]
x = np.percentile(Marks, 45)
print (x)
Output -
62.8
As it shown, it returned a value that shows how many marks students got above 45%.
Data Distribution
Earlier, we worked with data, but that was on a small scale, which means that data was just a bit of significant and realistic data in real-world problems.
It is tedious to work with big data in the early stage of small projects.
So how to get big data?
Big Data Sets
To build large data sets, we use the Python module NumPy, which has a range of options to generate random data sets regardless of size.
Example: Create a set of 100 random numbers from 10 to 20.
import numpy as np
data_set = np.random.uniform(10, 20, 100)
print (data_set)
[13.38299205 18.01580669 17.02081562 11.77377081 12.69501008 18.76522416
13.91344119 13.60202607 12.15678422 14.11413323 13.21394625 16.54937797
13.59474679 14.26320092 13.80828941 16.34440104 10.39993553 15.46494338
10.41695753 12.76514056 11.72558636 19.02983381 10.66168355 11.76709056
15.60670045 17.72205624 18.95663704 15.5515067 11.90693495 16.31697104
11.4483667 15.11501072 11.52720814 11.2289283 15.12808577 12.61315005
14.39045934 10.85921454 11.30185564 15.31424129 19.27459975 10.70108656
19.94672168 16.39083343 13.21372947 10.80109336 13.00774192 18.62775174
14.4165342 12.11367457 15.7729824 19.65623126 16.19241096 16.11580766
12.26976805 19.34958711 14.08468552 13.09321088 17.67535161 17.96116085
10.68782391 17.55858529 18.32730636 11.86655782 13.03786586 10.41227579
14.56578666 11.27612942 17.48043824 11.97837879 15.50466613 14.48695143
19.79191699 13.12199032 18.88577316 10.23694902 17.68729269 18.28547097
14.05099738 11.09106986 19.53213855 14.32717191 18.45282231 18.28829096
19.65434796 13.94371623 12.72428148 13.46691759 13.28333032 17.16257679
14.86544913 12.85619305 10.39865977 13.98904176 14.56325333 14.96543929
15.15778962 13.39786431 10.01417788 12.51462504]
Histogram
To visualize the data set in the graph, we can use the Matplotlib method hist().
Note: Find out more about using the Matplotlib module by reading Matplotlib Tutorial.
Example: Draw a histogram with the given data set.
import numpy as np
import matplotlib.pyplot as plt
data_set = np.random.uniform(10, 20, 100)
plt.hist(data_set, 18)
plt.show()
As a result, it returned a Histogram created from given data.
Note: Data is generated randomly, therefore every time it will not return the same result.
Big Data Distributions
A set of 100 values does not consider a lot, but you can make an array of random values. By changing your parameters, you can make the data set as large as you’d like.
Example: Create a set of 1000 random numbers from 1 to 20.
import numpy as np
data_set = np.random.uniform(1, 20, 1000)
print (data_set)
[ 2.68284406 19.30694991 16.65383921 18.55471701 4.70346159 16.29699052
9.50559452 11.10950173 14.3041972 1.39679134 17.72094537 4.14102451
9.54203582 18.77060463 12.51891703 17.31634713 11.70200927 12.05415392
3.81738685 1.5661812 11.24821039 18.54222366 16.18801869 18.99189221
14.56934761 5.3644479 17.91362365 17.80043123 6.36233596 2.5723287
3.2887804 18.82218981 6.19916744 8.80655601 11.0755259 12.81059619
17.84134297 19.93284239 14.88767059 15.05756906 8.96185196 4.75425607
15.35962198 11.69880277 15.90197501 2.34193269 4.83790642 3.16225479
6.1547387 4.12802624 12.35229347 1.11053044 3.76303772 13.03854785
19.6487084 13.94900319 16.67858887 7.46180519 7.67090646 17.66687265
3.45180209 13.41481762 19.29474335 12.12074493 1.84056973 18.12877054
5.94005805 19.44342378 1.21762351 17.90821213 4.08002328 7.18451625
5.06869492 15.40131127 10.5710893 7.78646599 16.87135914 11.38047853
4.23085077 5.8045335 17.64617202 1.60025407 2.31686046 13.02668694
6.12200063 18.54691212 7.03539393 7.13276625 12.89408759 13.56044378
5.94902979 12.28058722 15.86691714 4.54260625 1.90325003 13.97100725
17.09954854 3.14509746 10.95124018 10.31550978 10.93273627 17.0859282
14.29736977 12.6945781 14.1120251 14.2543168 4.97813506 7.50472078
9.35401342 6.4704281 6.32950381 9.8485998 13.27325802 9.56066328
1.16270337 9.73355442 8.83447507 13.75864309 16.54317726 1.4462913
18.0424996 13.58435098 10.45178192 5.57316267 6.60111476 14.60948159
9.01016988 4.73155305 10.72594475 1.92208041 16.75167933 19.46995145
18.1144034 14.47006203 13.92000884 3.3824039 8.74174971 18.1798602
17.35194918 15.85792074 8.29637198 6.37516705 10.51614556 19.92357184
12.37006969 1.95233356 9.55441568 1.40142866 10.1312481 18.02089924
18.29029191 17.73409822 17.84251397 9.7447081 9.71481697 13.6548868
17.37030435 16.36110037 2.20253671 1.25945193 16.98632498 16.66871667
16.47734076 14.75418909 11.82011605 14.65770869 10.33160747 6.15865809
18.86801083 15.27174114 15.78001809 7.97126725 8.75705367 14.11080899
18.0901586 4.33671917 6.34645877 16.0543543 13.74162995 5.62893212
6.87519141 4.18667454 18.12897226 16.76950034 6.63657079 7.62916742
12.31055102 12.63054808 9.05282658 4.458712 16.03326673 8.9712681
14.37981465 13.01470915 11.77843574 5.0598084 8.56230647 7.62765179
18.08111907 16.00672054 11.85420408 7.55759042 17.56853919 5.87619219
14.82067182 16.53249367 16.14679673 8.81673644 2.36191931 7.91540339
14.60880143 18.25186613 1.35301916 1.39289951 7.19234846 8.59003438
10.65926329 11.56859105 13.3086017 10.53845096 9.05920321 2.38675077
13.98210117 12.27678627 2.11788438 10.56490776 8.1807473 10.23851933
12.1700239 12.70801787 1.70334211 10.04025611 7.03456016 10.96545274
13.36661346 12.01991377 4.47020019 5.25230198 18.55228793 2.9751601
19.30324689 4.70227332 3.78800656 5.65181784 6.27681342 2.29156551
4.85132641 2.2168593 19.18334034 1.84605864 5.68471505 5.82173105
18.9890535 9.99189014 13.19219269 6.00917318 10.66303158 16.59943689
18.49386956 6.86122869 4.65412214 2.83544282 14.39486846 2.79235635
17.98245437 14.63851702 2.86578784 3.30032668 3.81220202 9.99366174
8.033083 11.99681773 17.51653442 3.76386469 11.55797275 7.74000305
1.38022942 3.70198612 15.39116237 13.39837799 12.38785551 3.32810113
9.06933208 14.56616268 8.47560263 13.65045843 7.58276071 16.247074
19.64338362 11.79075262 5.58799464 15.83210538 14.43117395 11.5508438
5.82289929 2.30115398 5.61743903 13.72937397 14.41120782 17.15616908
15.74530842 18.03067512 9.8895974 5.10173368 14.9546928 9.42473319
18.14967249 6.17852932 18.62979844 13.81425162 10.40391681 5.04455942
19.65621511 8.64739888 15.06320431 17.58040548 15.57964799 5.39830662
17.04715289 6.7352639 12.07345064 2.56205838 8.79697532 7.15802427
12.44821443 16.45619984 16.72507094 4.70716806 1.48013735 5.20611648
12.15979943 3.18866054 16.47910834 16.28355109 3.62130467 3.07307956
2.60717192 17.08254354 8.10494809 10.58204015 18.39375232 14.53077326
7.49103107 12.29224884 6.65818839 8.44392058 5.19610276 8.16156943
11.90445797 17.21148527 14.39354418 1.47673518 6.85730247 12.94819688
7.30028159 16.09814415 8.01658236 5.70582793 14.05202239 3.07217961
5.51333652 9.79646816 12.73109336 13.42984753 1.19931686 18.98223597
9.21191132 10.77960153 3.44222073 5.91753139 8.9059537 17.81532335
11.26923825 1.70155757 8.94562283 4.11207134 10.24657421 10.90081654
3.25893502 2.69287545 3.04676386 9.46630252 18.27922935 2.15723553
14.88767479 19.53463582 11.43093946 8.28183849 5.40649948 17.94455735
1.78805073 1.04516752 7.06630742 14.34729831 4.52037893 12.81790365
16.03971273 4.14512564 18.92757643 6.73775376 12.69152368 9.60705336
5.96629972 11.92864593 6.73882302 18.42825044 19.44192096 13.98098841
19.18517718 12.65311457 1.26130143 10.36038209 13.79683437 10.52840618
7.92908896 16.22838485 12.38787933 12.44017955 12.07564921 19.16857589
1.72209157 8.45158243 19.29683279 1.90449683 8.16089726 8.41843022
12.60912041 16.56266832 3.87542206 14.65918682 8.95657421 6.8362758
11.43972523 11.53629186 1.1818171 2.89668459 11.55067712 19.71487259
13.53286144 18.25786524 6.83215967 13.19429236 12.41575259 3.90490732
7.79793911 1.35720961 19.38969549 16.68092726 6.39675592 11.58863663
1.52434281 17.76421562 7.01080559 5.98679707 19.6360701 19.02191209
11.86296726 6.66684482 18.01003076 12.9927335 11.78975425 14.44809825
9.76253463 11.23858045 3.47369978 11.57247884 13.41913773 4.01150967
13.88701186 8.13189203 2.26619129 17.04138505 2.96337124 13.72430942
10.11391851 18.13870581 4.07872642 18.96126303 6.05165301 1.62107589
6.27944621 11.63782584 15.87898477 13.50048893 10.42468209 14.32255352
3.56642721 5.39458348 3.60331924 10.6333141 19.91236931 11.12253267
16.95428674 14.37098124 17.05158391 9.24116609 3.72413343 11.15113696
2.31104941 15.44686824 1.35054295 9.54523556 15.5672758 11.55648213
19.61085311 18.96168866 10.35441923 13.53334166 4.69494122 5.45828648
8.99397961 9.88431995 17.40923859 2.53905286 14.10332414 4.77018695
6.92331657 8.87406083 14.9990223 19.2697055 18.31991966 10.335444
14.40082601 19.05141842 6.45326835 10.56246992 6.93588843 16.1575082
8.72344603 19.21693783 12.40084231 2.82880249 17.26081728 15.50571215
18.22913392 9.08525062 8.81517044 14.00295998 11.27110522 15.06006983
8.78592585 6.03519428 1.21862452 2.12559042 14.9782146 19.04203281
1.24332743 3.97096582 16.90648162 8.28166465 14.18988428 2.08693087
1.08651955 3.21052702 11.8330645 5.13361811 1.84292134 19.55180562
6.76788255 7.03221102 15.98148186 13.94630962 3.5548162 3.6278384
16.01803076 12.46003371 1.94028536 14.45309543 16.88674581 13.60143857
2.47334437 16.44452332 10.36331142 13.50185818 1.80504367 5.35045681
7.53205433 15.14150467 4.60184005 18.82810544 19.66101154 13.16138002
19.01445706 1.98284414 5.31535847 19.65592877 8.65797539 12.7629716
16.25685409 13.61716591 19.57035376 9.91142541 14.03565244 15.75965915
4.9445043 15.45547834 11.203243 9.8733124 1.9328347 12.58114448
6.879032 14.59125248 4.15563702 8.28933544 11.29932155 3.49515306
10.54992373 12.29487422 18.54678782 10.66959335 16.46020035 7.70054899
13.10847404 14.31281761 6.40356048 19.64755775 9.4222633 5.52483508
17.36450206 8.88394896 3.00485118 8.78415 9.67515373 7.51144372
2.02538234 4.72197569 9.68805904 17.97288966 13.80733266 14.28454767
7.68220899 3.40843569 9.24727294 6.57583007 2.8914721 19.59426975
11.60274023 11.39963997 3.26091623 18.52749383 14.73757444 1.21068702
7.08294279 18.00521277 2.78958391 9.79245563 4.78128938 14.70301425
11.75840885 16.25484715 6.90578491 15.02079385 14.28398209 17.46925102
18.56251637 14.52477152 4.93576374 2.72879774 18.97938336 6.85353327
1.49163371 2.6337953 17.18447156 15.01752815 12.40977831 1.52975153
17.34197869 3.43182961 1.57839746 11.73795199 19.05783612 19.71879662
11.37938967 16.08249135 13.19375133 19.21793169 17.99955706 17.20476262
17.43288283 3.51212476 3.64369151 10.51111751 10.77195893 10.37812739
4.12083327 6.58363575 1.82828506 7.55504627 9.12313459 17.32197239
9.45846971 1.32421291 9.13941967 13.93079524 10.82720674 4.59170739
9.67271158 17.37155406 17.56945832 4.10585577 11.86716721 8.9094759
5.25787425 11.41212783 12.29677281 12.5607613 1.56572299 16.13805135
6.63457627 14.58395642 11.59932162 8.65977715 11.17144656 9.92630729
15.66162148 7.82260476 10.22238646 5.19037597 18.82565311 15.9281575
16.53349542 1.22812144 18.44855757 13.64598686 9.54173728 11.01796179
6.32587344 5.94682566 13.65491177 15.28287223 16.92188191 13.27513146
8.34912492 12.93857362 11.56082266 9.53413789 14.70346224 9.83312831
6.68529342 15.56211145 16.88310286 14.9993651 4.56165993 8.00319503
4.83032029 18.52915637 8.81295345 6.48064353 3.75929371 4.91112693
17.09476815 5.2739131 4.69094164 7.82627319 14.52544194 16.25652485
4.99139303 14.48741318 15.28226039 13.0599178 3.01140484 16.37546923
8.16349377 7.77769467 7.20164244 4.87508143 2.04154432 15.30465129
4.47306262 17.21360493 8.59699346 9.77599858 1.45770273 2.7048445
11.92154927 8.89460821 9.78640363 12.81791229 11.28708905 8.28616898
16.96559147 8.54799823 18.86598749 19.78388517 15.10944487 12.09970784
11.27405175 10.57452053 7.7802434 19.19013378 3.29663787 1.72740293
7.38176408 13.0658844 13.01662051 14.06911427 9.01198737 12.56844029
14.71080473 18.51037596 11.59073208 3.10104498 3.69900617 12.82106544
14.63939889 15.75955821 8.4277436 10.05631337 11.24681532 13.56329425
7.65946352 2.17431344 3.05459262 18.24270356 17.67016885 1.70765799
5.82143697 3.821201 2.02213436 4.80726968 13.68237456 18.49932392
9.49306261 15.14684949 2.68074297 8.02049596 16.41188829 18.72110056
12.84778665 4.77533094 4.04047908 17.8435927 8.4660641 15.61991871
2.19808815 6.71597302 11.3959533 3.8019543 10.81602106 13.22017599
13.93689926 9.61999353 7.18364705 6.76681695 13.61586318 2.56682801
1.2635081 13.14878474 19.74097372 18.96720024 11.6213272 1.35385837
9.22516327 18.62260496 11.30850318 16.55786945 17.0795904 4.35594454
5.83906367 6.94655833 3.23484597 17.94922942 12.75849988 7.65597128
2.44667273 14.20888511 16.44046922 12.13289775 16.47074426 10.46082682
17.53321003 9.40250986 3.80957857 1.14144124 5.95104479 2.55214873
15.03623717 11.98151374 4.83774944 16.54707546 18.89287499 3.675759
6.8657436 6.99032026 7.99076721 4.87949426 19.46837191 3.79614568
6.72792438 11.26955338 6.90773764 6.33031443 11.34585115 14.81423048
19.12230506 16.78362203 9.04538296 8.34005977 16.70997302 17.9694551
19.14426204 2.99099817 18.26237007 7.91653762 13.8818763 9.96841786
11.62077927 6.27019828 11.75004855 3.26273425 13.86333844 4.40368066
6.43366134 17.98923619 14.73532524 14.07716539 10.19648463 14.55291085
3.97172266 3.20639077 16.8158471 16.6589792 11.63183583 15.11471195
2.85747805 16.43737151 11.25354336 14.25672552 13.4879228 13.83305666
11.83302899 14.53237581 19.46133221 11.14732109 9.9364367 19.10141919
19.84264944 10.82693011 13.59181113 14.12451847 18.80255227 12.91549099
17.21204908 13.4456572 14.21361558 13.42743164 16.0411952 3.8614885
6.79280165 3.20691356 9.28432956 11.11471783 6.88681969 4.34494846
15.33603516 4.01908672 10.1040883 11.29970739 4.51261244 18.09215055
3.61361207 15.59025818 17.302529 3.28711345 17.11885296 16.78262672
14.58830822 18.84395424 12.59180699 14.15110893 17.45168623 14.7809512
7.44724417 13.48323567 4.90653204 12.86433649 3.03764207 7.6844456
8.62670856 1.93745013 6.35474297 4.69304274 1.41573836 2.35272146
16.82016687 18.09218852 19.6064135 14.5920794 17.58456152 15.83111451
5.37509657 3.34028129 4.1456231 13.9866889 6.18081457 4.49961602
1.69223174 4.19981165 12.49173066 16.32761904]
Example: Draw a histogram with the given data set.
import numpy as np
import matplotlib.pyplot as plt
data_set = np.random.uniform(1, 20, 1000)
plt.hist(data_set, 250)
plt.show()
