1、1,Lecture 1 Sampling of Signals,by Graham C. Goodwin University of Newcastle Australia,Lecture 1 Presented at the “Zaborszky Distinguished Lecture Series” December 3rd, 4th and 5th, 2007,2,Recall Basic Idea of Sampling and Quantization,Quantization,Sampling,t1,t3,t2,t4,t,0,3,In this lecture we will
2、ignore quantization issues and focus on the impact of different sampling patterns for scalar and multidimensional signals,4,Outline,One Dimensional Sampling Multidimensional Sampling Sampling and Reciprocal Lattices Undersampled Signals Filter Banks Generalized Sampling Expansion (GSE) Recurrent Sam
3、pling Application: Video Compression at Source Conclusions,5,Sampling: Assume amplitude quantization sufficiently fine to be negligible.Question: Say we are given Under what conditions can we recover from the samples?,6,A Well Known Result (Shannons Reconstruction Theorem for Uniform Sampling),Consi
4、der a scalar signal f(t) consisting of frequency components in the range . If this signal is sampled at period , then the signal can be perfectly reconstructed from the samples using:,7,Low pass filter recovers original spectrum,Hence,or,Proof: Sampling produces folding,8,A Simple (but surprising) E
5、xtension,where,Recurrent Sampling,is a periodic sequence of integers; i.e.,Let,Note that the average sampling period is,e.g.,average 5,9,Non-uniform,Uniform,0,9,-1,10,19,20,x,x,x,x,x,x,0,5,10,15,20,x,x,x,x,x,10,Claim: Provided the signal is bandlimited to where , then the signal can be perfectly rec
6、onstructed from the periodic sampling pattern.where = average sampling periodProof:We will defer the proof to later when we will use it as an illustration of Generalized Sampling Expansion (GSE) Theorem.,11,Outline,One Dimensional Sampling Multidimensional Sampling Sampling and Reciprocal Lattices U
7、ndersampled Signals Filter Banks Generalized Sampling Expansion (GSE) Recurrent Sampling Application: Video Compression at Source Conclusions,12,Multidimensional Signals,Digital Photography,Digital Video,x1,x2,x1,x2,x3 (time),13,Outline,One Dimensional Sampling Multidimensional Sampling Sampling and
8、 Reciprocal Lattices Undersampled Signals Filter Banks Generalized Sampling Expansion (GSE) Recurrent Sampling Application: Video Compression at Source Conclusions,14,How should we define sampling for multi-dimensional signals?Utilize idea of Sampling LatticeSampling Lattice,15,Also, need multivaria
9、ble frequency domain concepts.These are captured by two ideas Reciprocal Lattice Unit Cell,16,Unit Cell (Non-unique),Reciprocal Lattice,17,One Dimensional Example,Sampling Lattice,0,-20,10,20,x,x,x,x,-10,x,18,Reciprocal Lattice and Unit Cell,Unit Cell,0,19,Multidimensional Example,x1,x2,1 2 3 4 5,-4
10、 -3 -2 -1,-1 -2 -3 -4,5 4 3 2 1,20,Reciprocal Lattice and Unit Cell for Example,1/4 1/2 3/4 1,-1/4 -1/2 -3/4 -1,1/2 1/4,21,Outline,One Dimensional Sampling Multidimensional Sampling Sampling and Reciprocal Lattices Undersampled Signals Filter Banks Generalized Sampling Expansion (GSE) Recurrent Samp
11、ling Application: Video Compression at Source Conclusions,22,We will be interested here in the situation where the Sampling Lattice is not a Nyquist Lattice for the signal (i.e., the signal cannot be perfectly reconstructed from the original pattern!),Strategy: We will generate other samples by filt
12、ering or shifting operations on the original pattern.,23,Consider a bandlimited signal .Assume the D-dimension Fourier transform has finite support, S.Then for given D-dimensional lattice T, there always exists a finite set , such that support,Heuristically: The idea of “Tiling” the area of interest
13、 in the frequency domain,24,One Dimensional Example,Our one dimensional example continued. Sampling Lattice,Unit Cell,0,Bandlimited spectrum,Use,Support,25,Outline,One Dimensional Sampling Multidimensional Sampling Sampling and Reciprocal Lattices Undersampled Signals Filter Banks Generalized Sampli
14、ng Expansion (GSE) Recurrent Sampling Application: Video Compression at Source Conclusions,26,Generation of Extra Samples,Suppose now we generate a data setas shown in below,Q Channel Filter Bank,27,Outline,One Dimensional Sampling Multidimensional Sampling Sampling and Reciprocal Lattices Undersamp
15、led Signals Filter Banks Generalized Sampling Expansion (GSE) Recurrent Sampling Application: Video Compression at Source Conclusions,28,Define,Let,be the solution (if it exists) of,for,29,Conditions for Perfect Reconstruction,can be reconstructed fromif and only if has full row rank for all in the
16、Unit Cell where,GSE Theorem:,30,Proof:Multiply both sides by where (the Reciprocal Lattice). Then sum over qNote that “tiles” the entire support S Thus,from the Matrix identity that defines,31,where we have used the fact that Since is the output of f(x) passing through , thenHence, we finally have,3
17、2,Outline,One Dimensional Sampling Multidimensional Sampling Sampling and Reciprocal Lattices Undersampled Signals Filter Banks Generalized Sampling Expansion (GSE) Recurrent Sampling Application: Video Compression at Source Conclusions,33,Special Case: Recurrent Sampling,(where is implemented by a
18、“spatial” shift )This amounts to the sampling pattern:where w.l.o.g.Now, given the samples , our goal is to perfectly reconstruct,34,Here , and ThusTo apply the GSE Theorem we require,Nonsingular,35,Something to think about,The GSE result depends on inversion of a particular matrix, H(w). Of course
19、we have assumed here perfect representation of all coefficients. An interesting question is what happens when the representation is imperfect i.e. coefficients are represented with finite wordlength (i.e. they are quantized) We will not address this here but it is something to keep in mind.,36,Retur
20、n to our one-dimensional example,Recall that we had so that supportSay we use recurrent sampling with,37,0,10,20,x,x,x,0,9,19,x,x,x,0,x,x,x,-1,-1,x,x,19 20,x,9 10,38,Condition for Perfect Reconstruction is,nonsingular,Hence, the original signal can be recovered from the sampling pattern given in the
21、 previous slide.,39,Summary,We have seen that the well known Shannon reconstruction theorem can be extended in several directions; e.g. Multidimensional signals Sampling on a lattice Recurrent sampling Given specific frequency domain distributions, these can be matched to appropriate sampling patter
22、ns.,40,Outline,One Dimensional Sampling Multidimensional Sampling Sampling and Reciprocal Lattices Undersampled Signals Filter Banks Generalized Sampling Expansion (GSE) Recurrent Sampling Application: Video Compression at Source Conclusions,41,Application: Video Compression Source,Introduction to v
23、ideo cameras Instead of tape, digital cameras use 2D sensor array (CCD or CMOS),42,Image Sensor,A 2D array of sensors replaces the traditional tapeEach sensor records a point of the continuous imageThe whole array records the continuous image at a particular time instant,43,2D Colours Sensor Array,D
24、ata transfer from array is sequential and has a maximal rate of Q.,* Based on http:/ 3D sampling a sequence of identical frames equally spaced in time,Current Technology,45,The volume of box depends on the capacity:pixel rate = (frame rate) x (spatial resolution),Video Bandwidth,depends on spatial r
25、esolution of the frames,depends on the frame rate,46,Data recording on sensor:,Sensor array density - for spatial resolution,Sensor exposure time - for frame rate,2. Data reading from sensor:,Data readout time- for pixel rate,Hard Constraints,47,Generally Q RF Need: R1 R F1 F s.t. R1F1 = Q,Compromis
26、e: spatial resolution R1 R temporal resolution F1 F,BUT.,48,volume determinedby,Actual Capacity (Data Readout),49,Observation,Most energy of typical video scene is concentrated around the plane and the axis.,50,uniform sampling- compromise in frame rate,uniform sampling- compromise in spatial resolu
27、tion,uniform sampling - no compromise,The Spectrum of this Video Clip,51,New Idea,Idea is to deviate from constant resolutions in a recorded video clip. This means that sampling patterns within the video clip will not be uniform. Specifically, the idea is to have, within the recorded video clip, a c
28、ombination of fast frames with low spatial resolution and slow frames with high spatial resolution.,52,frame type A,frame type B,Recurrent Non-Uniform Sampling,53,What Does it Buy?,54,Schematic Implementation,non-uniform data from the sensor,uniform high def. video,compression at the source,55,Recur
29、rent Non-Uniform Sampling,A special case of Generalized Sampling Expansion Theorem,56,Sampling Pattern,The resulting sampling pattern is given by:,57,Frequency Domain,where:,is the unit cell of the reciprocal lattice,58,Reciprocal Lattice,x,t,59,Apply the GSE Theorem,where: is uniquely defined by H1
30、H2() is a set of 2(L+M)+1 constraints,If exists, we can find the reconstruction function,60,Reconstruction Scheme,H2L+1,2L+1,I(x,t),(x,t),The sub-sampled frequency of each filter H is:,61,Reconstruction functions,for r = 2,3,2L+1,for r = 2(L+1),2(L+M)+1,Multidimensional sinc like functions,62,Demo,F
31、ull resolution sequence,Reconstructed sequence,Temporal decimation,Spacial decimation,63,Outline,One Dimensional Sampling Multidimensional Sampling Sampling and Reciprocal Lattices Undersampled Signals Filter Banks Generalized Sampling Expansion (GSE) Recurrent Sampling Application: Video Compressio
32、n at Source Conclusions,64,Conclusions,Nonuniform sampling of scalar signals Nonuniform sampling of multidimensional signals Generalized sampling expansion Application to video compression A remaining problem is that of joint design of sampling schemes and quantization strategies to minimize error f
33、or a given bit rate,65,References,One Dimensional SamplingA. Feuer and G.C. Goodwin, Sampling in Digital Signal Processing and Control. Birkhuser, 1996.R.J. Marks II, Ed., Advanced Topics in Shannon Sampling and Interpolation Theory. New Your: Springer-Verlag, 1993.Multidimensional SamplingW.K. Prat
34、t, Digital Image Processing, 3rd ed: John Wiley & Sons, 2001.B.L. Evans, “Designing commutative cascades of multidimensional upsamplers and downsamplers,” IEEE Signal Process Letters, Vol4, No.11, pp.313-316, 1997.Sampling and Reciprocal Lattices, Undersampled SignalsA.Feuer, G.C. Goodwin, Reconstru
35、ction of Multidimensional Bandlimited Signals for Uniform and Generalized Samples, IEEE Transactions on Signal Processing, Vol.53, No.11, 2005.A.K. Jain, Fundamentals of Digital Image Processing, Englewood Cliffs, NJ: Prentice-Hall, 1989.,66,References,Filter BanksY.C. Eldar and A.V. Oppenheim, Filt
36、erbank reconstruction of bandlimited signals from nonuniform and generalized samples, IEEE Transactions on Signal Processing, Vol.48, No.10, pp.2864-2875, 2000.P.P. Vaidyanathan, Multirate Systems and Filter Banks. Englewood Cliffs, NJ: Prentice-Hall, 1993.H. Blceskei, F. Hlawatsch and H.G. Feichtin
37、ger, Frame-theoretic analysis of oversampled filter banks, IEEE Transactions on Signal Processing, Vol.46, No.12, pp.3256-3268, 1998.M. Vetterli and J. Kovaevi, Wavelets and Subband Coding, Englewood Cliffs, NJ: Prentice Hall, 1995.,67,References,Generalized Sampling Expansions, Recurrent SamplingA.
38、 Papoulis, Generalized sampling expansion, IEEE Transaction on Circuits and Systems, Vol.CAS-24, No.11, pp.652-654, 1977.A. Feuer, On the necessity of Papoulis result for multidimensional (GSE), IEEE Signal Processing Letters, Vol.11, No.4, pp.420-422, 2004.K.F.Cheung, A multidimensional extension o
39、f Papoulis generalized sampling expansion with application in minimum density sampling, in Advanced Topics in Shannon Sampling and Interpolation Theory, R.J. Marks II. Ed., New York: Springer-Verlag, pp.86-119, 1993.Video Compression at SourceE. Shechtman, Y. Caspi and M. Irani, Increasing space-tim
40、e resolution in video, European Conference on Computer Vision (ECCV), 2002.N. Maor, A. Feuer and G.C. Goodwin, Compression at the source of digital video, To appear EURASIP Journal on Applied Signal Processing.,68,Lecture 1 Sampling of Signals,by Graham C. Goodwin University of Newcastle Australia,Lecture 1 Presented at the “Zaborszky Distinguished Lecture Series” December 3rd, 4th and 5th, 2007,
