ó ʽ÷Xc@`s`dZddlmZmZmZdZddddgZddlZddlZ dd l m Z dd l m Z d d lmZd efd„ƒYZed„Zed„Zed„Zd„Zed„Zdd„Zdded„Zd„Zd„Zied6ed6Zd„Zd„Zied6ed6Zd ddded!„Z d"„Z!dS(#s0 ==================================================================== K-means clustering and vector quantization (:mod:`scipy.cluster.vq`) ==================================================================== Provides routines for k-means clustering, generating code books from k-means models, and quantizing vectors by comparing them with centroids in a code book. .. autosummary:: :toctree: generated/ whiten -- Normalize a group of observations so each feature has unit variance vq -- Calculate code book membership of a set of observation vectors kmeans -- Performs k-means on a set of observation vectors forming k clusters kmeans2 -- A different implementation of k-means with more methods -- for initializing centroids Background information ====================== The k-means algorithm takes as input the number of clusters to generate, k, and a set of observation vectors to cluster. It returns a set of centroids, one for each of the k clusters. An observation vector is classified with the cluster number or centroid index of the centroid closest to it. A vector v belongs to cluster i if it is closer to centroid i than any other centroids. If v belongs to i, we say centroid i is the dominating centroid of v. The k-means algorithm tries to minimize distortion, which is defined as the sum of the squared distances between each observation vector and its dominating centroid. Each step of the k-means algorithm refines the choices of centroids to reduce distortion. The change in distortion is used as a stopping criterion: when the change is lower than a threshold, the k-means algorithm is not making sufficient progress and terminates. One can also define a maximum number of iterations. Since vector quantization is a natural application for k-means, information theory terminology is often used. The centroid index or cluster index is also referred to as a "code" and the table mapping codes to centroids and vice versa is often referred as a "code book". The result of k-means, a set of centroids, can be used to quantize vectors. Quantization aims to find an encoding of vectors that reduces the expected distortion. All routines expect obs to be a M by N array where the rows are the observation vectors. The codebook is a k by N array where the i'th row is the centroid of code word i. The observation vectors and centroids have the same feature dimension. As an example, suppose we wish to compress a 24-bit color image (each pixel is represented by one byte for red, one for blue, and one for green) before sending it over the web. By using a smaller 8-bit encoding, we can reduce the amount of data by two thirds. Ideally, the colors for each of the 256 possible 8-bit encoding values should be chosen to minimize distortion of the color. Running k-means with k=256 generates a code book of 256 codes, which fills up all possible 8-bit sequences. Instead of sending a 3-byte value for each pixel, the 8-bit centroid index (or code word) of the dominating centroid is transmitted. The code book is also sent over the wire so each 8-bit code can be translated back to a 24-bit pixel value representation. If the image of interest was of an ocean, we would expect many 24-bit blues to be represented by 8-bit codes. If it was an image of a human face, more flesh tone colors would be represented in the code book. i(tdivisiontprint_functiontabsolute_importtrestructuredtexttwhitentvqtkmeanstkmeans2N(t_asarray_validated(t _numpy_compati(t_vqt ClusterErrorcB`seZRS((t__name__t __module__(((s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyR [scC`sdt|d|ƒ}tj|ddƒ}|dk}|jƒr\d||>> # f0 f1 f2 >>> obs = [[ 1., 1., 1.], #o0 ... [ 2., 2., 2.], #o1 ... [ 3., 3., 3.], #o2 ... [ 4., 4., 4.]] #o3 check_finite : bool, optional Whether to check that the input matrices contain only finite numbers. Disabling may give a performance gain, but may result in problems (crashes, non-termination) if the inputs do contain infinities or NaNs. Default: True Returns ------- result : ndarray Contains the values in `obs` scaled by the standard deviation of each column. Examples -------- >>> from scipy.cluster.vq import whiten >>> features = np.array([[1.9, 2.3, 1.7], ... [1.5, 2.5, 2.2], ... [0.8, 0.6, 1.7,]]) >>> whiten(features) array([[ 4.17944278, 2.69811351, 7.21248917], [ 3.29956009, 2.93273208, 9.33380951], [ 1.75976538, 0.7038557 , 7.21248917]]) t check_finitetaxisigð?sWSome columns have standard deviation zero. The values of these columns will not change.(RtnptstdtanytwarningstwarntRuntimeWarning(tobsRtstd_devt zero_std_mask((s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyR_s-     cC`s²t|d|ƒ}t|d|ƒ}tj||ƒ}|j|dtƒ}|j|krl|j|ƒ}n|}|tjtjfkrŸtj ||ƒ}nt ||ƒ}|S(sj Assign codes from a code book to observations. Assigns a code from a code book to each observation. Each observation vector in the 'M' by 'N' `obs` array is compared with the centroids in the code book and assigned the code of the closest centroid. The features in `obs` should have unit variance, which can be achieved by passing them through the whiten function. The code book can be created with the k-means algorithm or a different encoding algorithm. Parameters ---------- obs : ndarray Each row of the 'M' x 'N' array is an observation. The columns are the "features" seen during each observation. The features must be whitened first using the whiten function or something equivalent. code_book : ndarray The code book is usually generated using the k-means algorithm. Each row of the array holds a different code, and the columns are the features of the code. >>> # f0 f1 f2 f3 >>> code_book = [ ... [ 1., 2., 3., 4.], #c0 ... [ 1., 2., 3., 4.], #c1 ... [ 1., 2., 3., 4.]] #c2 check_finite : bool, optional Whether to check that the input matrices contain only finite numbers. Disabling may give a performance gain, but may result in problems (crashes, non-termination) if the inputs do contain infinities or NaNs. Default: True Returns ------- code : ndarray A length M array holding the code book index for each observation. dist : ndarray The distortion (distance) between the observation and its nearest code. Examples -------- >>> from numpy import array >>> from scipy.cluster.vq import vq >>> code_book = array([[1.,1.,1.], ... [2.,2.,2.]]) >>> features = array([[ 1.9,2.3,1.7], ... [ 1.5,2.5,2.2], ... [ 0.8,0.6,1.7]]) >>> vq(features,code_book) (array([1, 1, 0],'i'), array([ 0.43588989, 0.73484692, 0.83066239])) Rtcopy( RRt common_typetastypetFalsetdtypetfloat32tfloat64R Rtpy_vq(Rt code_bookRtcttc_obst c_code_booktresults((s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyR—s:c C`s|t|d|ƒ}t|d|ƒ}tj|ƒdkrvtj|ƒtj|ƒksftdƒ‚q‹t||ƒSntj|ƒ\}}tj|ƒtj|ƒks¸tdƒ‚n3||jdksëtd|jd|fƒ‚ntj|dtƒ}tj|ƒ}xWt|ƒD]I}tj |||ddƒ}tj |ƒ||<|||||c C`sétj|dtƒ}g}tj}x´||krÚ|jd}t||ƒ\}}|jtj|ddƒƒ||kr°tj |||ƒ\}} |j | ddƒ}nt |ƒdkr'|d|d}q'q'W||dfS(sM "raw" version of k-means. Returns ------- code_book the lowest distortion codebook found. avg_dist the average distance a observation is from a code in the book. Lower means the code_book matches the data better. See Also -------- kmeans : wrapper around k-means Examples -------- Note: not whitened in this example. >>> from numpy import array >>> from scipy.cluster.vq import _kmeans >>> features = array([[ 1.9,2.3], ... [ 1.5,2.5], ... [ 0.8,0.6], ... [ 0.4,1.8], ... [ 1.0,1.0]]) >>> book = array((features[0],features[2])) >>> _kmeans(features,book) (array([[ 1.7 , 2.4 ], [ 0.73333333, 1.13333333]]), 0.40563916697728591) RiRiÿÿÿÿiiþÿÿÿ( RtarraytTruetinfR)RtappendtmeanR tupdate_cluster_meanstcompresstlen( RtguesstthreshR!tavg_distR=R9tobs_codetdistortt has_members((s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyt_kmeans‚s!   icC`sØt|d|ƒ}t|ƒdkr3tdƒ‚nd }d }yt|ƒ}Wn#tk rtt|d|ƒ}nX|d k r»|jdkr£td|ƒ‚nt||d|ƒ}n||krÖtdƒ‚ntj}|j d} |}|dkr tdƒ‚nx¸t |ƒD]ª} tj j d| |ƒ} tj tj| d tƒddkƒrytj j| ƒ| } ntj|| dƒ}t||d|ƒ\} } | |kr| }| }qqW||f}|S( s Performs k-means on a set of observation vectors forming k clusters. The k-means algorithm adjusts the centroids until sufficient progress cannot be made, i.e. the change in distortion since the last iteration is less than some threshold. This yields a code book mapping centroids to codes and vice versa. Distortion is defined as the sum of the squared differences between the observations and the corresponding centroid. Parameters ---------- obs : ndarray Each row of the M by N array is an observation vector. The columns are the features seen during each observation. The features must be whitened first with the `whiten` function. k_or_guess : int or ndarray The number of centroids to generate. A code is assigned to each centroid, which is also the row index of the centroid in the code_book matrix generated. The initial k centroids are chosen by randomly selecting observations from the observation matrix. Alternatively, passing a k by N array specifies the initial k centroids. iter : int, optional The number of times to run k-means, returning the codebook with the lowest distortion. This argument is ignored if initial centroids are specified with an array for the ``k_or_guess`` parameter. This parameter does not represent the number of iterations of the k-means algorithm. thresh : float, optional Terminates the k-means algorithm if the change in distortion since the last k-means iteration is less than or equal to thresh. check_finite : bool, optional Whether to check that the input matrices contain only finite numbers. Disabling may give a performance gain, but may result in problems (crashes, non-termination) if the inputs do contain infinities or NaNs. Default: True Returns ------- codebook : ndarray A k by N array of k centroids. The i'th centroid codebook[i] is represented with the code i. The centroids and codes generated represent the lowest distortion seen, not necessarily the globally minimal distortion. distortion : float The distortion between the observations passed and the centroids generated. See Also -------- kmeans2 : a different implementation of k-means clustering with more methods for generating initial centroids but without using a distortion change threshold as a stopping criterion. whiten : must be called prior to passing an observation matrix to kmeans. Examples -------- >>> from numpy import array >>> from scipy.cluster.vq import vq, kmeans, whiten >>> features = array([[ 1.9,2.3], ... [ 1.5,2.5], ... [ 0.8,0.6], ... [ 0.4,1.8], ... [ 0.1,0.1], ... [ 0.2,1.8], ... [ 2.0,0.5], ... [ 0.3,1.5], ... [ 1.0,1.0]]) >>> whitened = whiten(features) >>> book = np.array((whitened[0],whitened[2])) >>> kmeans(whitened,book) (array([[ 2.3110306 , 2.86287398], # random [ 0.93218041, 1.24398691]]), 0.85684700941625547) >>> from numpy import random >>> random.seed((1000,2000)) >>> codes = 3 >>> kmeans(whitened,codes) (array([[ 2.3110306 , 2.86287398], # random [ 1.32544402, 0.65607529], [ 0.40782893, 2.02786907]]), 0.5196582527686241) Risiter must be at least 1.s)Asked for 0 cluster ? initial book was %sRHs0if k_or_guess is a scalar, it must be an integerisAsked for 0 cluster ? t return_countsN(RR+R'tNonet TypeErrorR7RMRRAR)R,trandomtrandintRR tuniqueR@t permutationttake(Rt k_or_guesstiterRHRtkRGtresultt best_disttNoR4tk_random_indicestbookR5t best_book((s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyRµsB_          cC`s^|jdkr|jd}n |j}tjj|ƒ}||| dd…fjƒ}|S(sÑPick k points at random in data (one row = one observation). This is done by taking the k first values of a random permutation of 1..N where N is the number of observation. Parameters ---------- data : ndarray Expect a rank 1 or 2 array. Rank 1 are assumed to describe one dimensional data, rank 2 multidimensional data, in which case one row is one observation. k : int Number of samples to generate. iiN(R&R)R7RRQRTR(tdataRXR0tptx((s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyt_kpointsAs   c`s„‡fd†}‡fd†}‡fd†}tj|ƒ}|dkrR||ƒS|jd|jdkrv||ƒS||ƒSdS(súReturns k samples of a random variable which parameters depend on data. More precisely, it returns k observations sampled from a Gaussian random variable which mean and covariances are the one estimated from data. Parameters ---------- data : ndarray Expect a rank 1 or 2 array. Rank 1 are assumed to describe one dimensional data, rank 2 multidimensional data, in which case one row is one observation. k : int Number of samples to generate. c`sQtj|ƒ}tj|ƒ}tjjˆƒ}|tj|ƒ9}||7}|S(N(RRCtcovRQtrandnR/(R_tmuRcRa(RX(s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyt init_rank1ls  c`sqtj|dƒ}tjtj|ddƒƒ}tjjˆ|jƒ}tj|tjj |ƒj ƒ|}|S(Nitrowvar( RRCt atleast_2dRcRQRdR7tdottlinalgtcholeskytT(R_ReRcRa(RX(s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyt init_ranknts %c`sžtj|ddƒ}tjj||dtƒ\}}}tjjˆ|jƒ}|dd…df|tj |j ddƒ}tj ||ƒ|}|S(NRit full_matricesi( RRCRjtsvdRRQRdR7ROR/R)Ri(R_Ret_tstvhRatsVh(RX(s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyt init_rank_def~s %2iiN(RR&R)(R_RXRfRmRttnd((RXs//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyt _krandinit\s     RQtpointscC`stjdƒdS(sPrint a warning when called.sKOne of the clusters is empty. Re-run kmean with a different initialization.N(RR(((s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyt _missing_warn’scC`stdƒ‚dS(s!raise a ClusterError when called.sKOne of the clusters is empty. Re-run kmean with a different initialization.N(R (((s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyt_missing_raise˜sRtraisei c C`sAt|d|ƒ}|tkr7tdt|ƒƒ‚ntj|ƒ}|dkr[d}n(|dkrw|jd}n tdƒ‚tj|ƒdkr§tdƒ‚ntj|ƒdksÈ|dkr@|tj|ƒksìtdƒ‚n|dkrt|ƒ} n*|j\} } | |ks1td ƒ‚n|j ƒ} nÂyt |ƒ} Wn't k rytd t|ƒƒ‚nX| dkrŸtd t|ƒƒ‚n| |ks»t j d ƒnyt|} Wn'tk ròtd t|ƒƒ‚nX| ||ƒ} t |ƒdkr'td|ƒ‚nt|| || t|ƒS(s… Classify a set of observations into k clusters using the k-means algorithm. The algorithm attempts to minimize the Euclidian distance between observations and centroids. Several initialization methods are included. Parameters ---------- data : ndarray A 'M' by 'N' array of 'M' observations in 'N' dimensions or a length 'M' array of 'M' one-dimensional observations. k : int or ndarray The number of clusters to form as well as the number of centroids to generate. If `minit` initialization string is 'matrix', or if a ndarray is given instead, it is interpreted as initial cluster to use instead. iter : int, optional Number of iterations of the k-means algrithm to run. Note that this differs in meaning from the iters parameter to the kmeans function. thresh : float, optional (not used yet) minit : str, optional Method for initialization. Available methods are 'random', 'points', and 'matrix': 'random': generate k centroids from a Gaussian with mean and variance estimated from the data. 'points': choose k observations (rows) at random from data for the initial centroids. 'matrix': interpret the k parameter as a k by M (or length k array for one-dimensional data) array of initial centroids. missing : str, optional Method to deal with empty clusters. Available methods are 'warn' and 'raise': 'warn': give a warning and continue. 'raise': raise an ClusterError and terminate the algorithm. check_finite : bool, optional Whether to check that the input matrices contain only finite numbers. Disabling may give a performance gain, but may result in problems (crashes, non-termination) if the inputs do contain infinities or NaNs. Default: True Returns ------- centroid : ndarray A 'k' by 'N' array of centroids found at the last iteration of k-means. label : ndarray label[i] is the code or index of the centroid the i'th observation is closest to. RsUnkown missing method: %siisInput of rank > 2 not supportedsInput has 0 items.tmatrixs/k is not an int and has not same rank than datasFk is not an int and has not same rank than datas,k (%s) could not be converted to an integer s#kmeans2 for 0 clusters ? (k was %s)s$k was not an integer, was converted.sunknown init method %ss9iter = %s is not valid. iter must be a positive integer.(Rt_valid_miss_methR'tstrRR&R)R7RFRR+RPRRt_valid_init_methtKeyErrort_kmeans2( R_RXRWRHtminittmissingRRuR1R9tdctclusterstinit((s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pyR sJ<     !       c C`s{xnt|ƒD]`}t||ƒd}tj|||ƒ\}}|jƒsg|ƒ||||RMRRbRvR~RxRyR|RR€(((s//tmp/pip-build-7oUkmx/scipy/scipy/cluster/vq.pytDs2   8 L F ! 8 3Œ  3   o