12 |
\begin{abstract} |
\begin{abstract} |
13 |
We demonstrate a simple but counterintuitive hardware-accelerated |
We demonstrate a simple but counterintuitive hardware-accelerated |
14 |
rendering trick for improving image quality. |
rendering trick for improving image quality. |
15 |
When rendering an isotropically textured 2 1/2D scene, |
When rendering isotropically textured polygons (using 2D rotations and isotropic |
16 |
|
scaling but no 3D rotations or shearing), |
17 |
stretching an image when putting it into a texture |
stretching an image when putting it into a texture |
18 |
and squishing it with texture coordinates when rendering yields |
and squishing it with texture coordinates when rendering yields |
19 |
a clearly better |
a clearly better |
20 |
image quality than simple trilinear filtering when hardware |
image quality than simple trilinear filtering when hardware |
21 |
anisotropic filtering is enabled. |
anisotropic filtering is enabled. |
22 |
We show a simple way to understand why this trick works, and discuss |
We show a simple way to understand why this trick works, based |
23 |
various generalizations. We show examples of text rendered both ways. |
on filter footprints. |
24 |
|
|
25 |
To show how our demonstration figures were |
In an appendix we show how our figures showing the actual texel samples used |
26 |
generated, |
by the hardware in different situations were generated, |
27 |
in an appendix we show how to probe and visualize the texture filtering happening |
paying |
|
on an actual hardware graphics accelerator in a general way, paying |
|
28 |
attention to what assumptions have to be made for such probing to work. |
attention to what assumptions have to be made for such probing to work. |
29 |
|
|
30 |
|
|
34 |
\section{Introduction} |
\section{Introduction} |
35 |
|
|
36 |
Recently, we were looking at increasing the resolution of the page |
Recently, we were looking at increasing the resolution of the page |
37 |
textures in our system which shows PDF files in a fisheye view, using |
textures in our FenPDF system which shows PDF files in a fisheye view, using |
38 |
2048x2048 textures for the pages. |
2048x2048 textures for the pages. |
39 |
Hardware anisotropic filtering was |
Hardware anisotropic filtering was |
40 |
enabled in order for the fisheye transformation not to blur the textures. |
enabled in order for the fisheye transformation not to blur the textures. |
41 |
The pages were approximately |
The pages were approximately |
42 |
letter-size and already scaled vertically nearly to the maximum size, |
letter-size and already scaled vertically nearly to the maximum size, |
43 |
but scaled isotropically. We decided to try to scale both axes to |
but scaled isotropically in the texture. |
44 |
|
We decided to try to scale both axes to |
45 |
the maximum extent, anisotropically, even though we suspected it might |
the maximum extent, anisotropically, even though we suspected it might |
46 |
degrade the image quality. |
degrade the image quality. |
47 |
|
|
48 |
However, it did not. To our great surprise, the image quality (the |
However, it did not. To our great surprise, the image quality (the |
49 |
readability of the text) actually \emph{improved significantly}. |
readability of the text) actually \emph{improved significantly}. |
50 |
Stretching the text in the texture and squishing it back with adjusting |
Stretching the text in the texture and squishing it back with adjusting |
51 |
the texture coordinates improved image quality. |
the texture coordinates improved image quality. In this article, we show |
52 |
|
why this is the case and how easy it is to take advantage of this effect. |
|
The most important area for reading is naturally the center of the fisheye, |
|
|
where the transformation is nearly orthonormal - the |
|
|
edges are mostly used for getting a sense of the context, not for reading. |
|
|
|
|
|
After some investigation, we discovered that we had found a special case |
|
|
of a general principle: if a texture image is only transformed through |
|
|
rotation and isotropic scaling, a better filtering result is always obtained |
|
|
by applying the stretch-squish operation. |
|
53 |
|
|
54 |
The rest of the article is organized as follows. |
The rest of the article is organized as follows. |
55 |
In Section~\ref{secrelated}, we discuss related work |
In Section~\ref{secrelated}, we discuss related work, i.e., texture filtering in general. |
|
and texture filtering in general, using pixel footprint in screen space (PFSS) |
|
|
diagrams to explain different filtering methods. |
|
56 |
In Section~\ref{secsquish}, we show why the stretch-squish method improves |
In Section~\ref{secsquish}, we show why the stretch-squish method improves |
57 |
image quality and its relation to other filtering methods. |
image quality and its relation to other filtering methods. |
|
In Section~\ref{seccomp}, we compare the performance of different filtering |
|
|
methods, including trilinear, stretch-squish aniso and supersampling, on a test image. |
|
58 |
Finally, we conclude. In Appendix A, we show how PFSS diagrams can be generated |
Finally, we conclude. In Appendix A, we show how PFSS diagrams can be generated |
59 |
in an elegant fashion by probing the hardware accelerators' true filtering behaviour. |
in an elegant fashion by probing the hardware accelerators' true filtering behaviour. |
60 |
|
|
67 |
% \subsection{The texture mapping primitive} |
% \subsection{The texture mapping primitive} |
68 |
|
|
69 |
Texture mapping is a ubiquitous ... \cite{heckbert86survey,haeberli93texture} |
Texture mapping is a ubiquitous ... \cite{heckbert86survey,haeberli93texture} |
70 |
originally introduced introduced by Catmull\cite{catmull74}. |
originally introduced by Catmull\cite{catmull74}. |
71 |
|
|
72 |
In off-line rendering, off |
In off-line rendering, off |
73 |
|
|
74 |
- off-line rendering: EWA XXXREF |
- off-line rendering: EWA XXXREF |
75 |
|
|
|
In our investigations for this article, we found the pixel footprint |
|
|
diagrams in screen space (PFSS) diagrams most useful for understanding |
|
|
the properties of a filtering method w.r.t.~anisotropy. |
|
|
Figure~\ref{figfootprint} shows a legend of PFSS diagrams and |
|
|
a diagram for the EWA filtering method. |
|
|
Appendix A shows how |
|
|
PFSS diagrams can be generated easily to show the actual behaviour of a hardware accelerator. |
|
|
|
|
|
\begin{figure} |
|
|
\begin{tabular*}{\columnwidth}{lc} |
|
|
a) & \\ |
|
|
& \includegraphics[width=5cm]{footprint.1} \\ |
|
|
b) & \\ |
|
|
& \includegraphics[width=4cm]{footprint.2} \\ |
|
|
\end{tabular*} |
|
|
\caption{ |
|
|
\label{figfootprint} |
|
|
Pixel footprint in screen space (PFSS) diagram. |
|
|
Texture samples' contribution to a pixel's value. |
|
|
a) An explanation of PFSS diagrams: the diagrams |
|
|
show the contribution of each texel to the pixel |
|
|
as a color (black = no contribution, white = large contribution). |
|
|
b) An example PFSS of an EWA texture filterer without |
|
|
mipmapping (mockup, just diagrammatic). |
|
|
In screen space, the |
|
|
filter is circular and has soft edges, while in texture space it would be |
|
|
elliptical. |
|
|
} |
|
|
\end{figure} |
|
|
|
|
76 |
% \subsection{Mipmapping: bi- and trilinear filtering} |
% \subsection{Mipmapping: bi- and trilinear filtering} |
77 |
|
|
78 |
|
On the other hand, in real-time rendering through |
79 |
|
hardware-accelerators, it is important that the number of samples can be |
80 |
|
kept constant regardless of the pixel footprint. |
81 |
|
|
82 |
- Trilinear/bilinear (mipmap) filtering was designed to avoid temporal and spatial aliasing\cite{williams83pyramidal} |
- Trilinear/bilinear (mipmap) filtering was designed to avoid temporal and spatial aliasing\cite{williams83pyramidal} |
83 |
|
|
84 |
- can blur sharp edges (text) too much |
For 3D rendering, the most well-known problem of |
85 |
- for 3D rendering, trilinear blurs when seen obliquely, \emph{anisotropically} |
trilinear filtering is that it blurs the texture the pixel footprint in texture space |
86 |
(squished more in one direction than in another). |
is far from round, i.e., in \emph{anisotropic} situation |
87 |
|
(squished more in one direction than in another). |
88 |
|
However, even in isotropic situations trilinear filtering |
89 |
|
can blur sharp edges (text) too much. |
90 |
|
|
91 |
|
- for sharp edges / small features, even under orthogonal transformations, trilinear bad: |
92 |
|
sampling at too low a resolution for much of the time! |
93 |
|
|
94 |
|
|
95 |
\begin{figure} |
- LOD bias sharpening causes spatial and temporal aliasing (flickering) |
96 |
a)\\ |
|
97 |
b)\\ |
\def\snapsize{3cm} |
98 |
c)\\ |
|
99 |
|
\begin{figure*} |
100 |
|
\begin{tabular}{rcrcrc} |
101 |
|
c) & |
102 |
|
\includegraphics[width=\snapsize]{snapsps/aniso-gf4go-nearest.ps} & |
103 |
|
c) & |
104 |
|
\includegraphics[width=\snapsize]{snapsps/aniso-gf4go-bilinear.ps} & |
105 |
|
c) & |
106 |
|
\includegraphics[width=\snapsize]{snapsps/aniso-gf4go-trilinear.ps} \\ |
107 |
|
c) & |
108 |
|
\includegraphics[width=\snapsize]{snapsps/aniso-gf4go-aniso-nearest.ps} & |
109 |
|
c) & |
110 |
|
\includegraphics[width=\snapsize]{snapsps/aniso-gf4go-trilinear-aniso.ps} & |
111 |
|
c) & |
112 |
|
\includegraphics[width=\snapsize]{snapsps/aniso-gf4go-aniso.ps} |
113 |
|
\end{tabular} |
114 |
\caption{ |
\caption{ |
115 |
\label{figbitrilinear} |
\label{figbitrilinear} |
116 |
Example PFSS diagrams: a) bilinear, b), c) trilinear filtering. |
PFSS (Pixel Footprint in Screen Space) diagrams generated on a Geforce4Go (NV17M), showing |
117 |
|
different types of filtering as they actually occur on the hardware |
118 |
|
(see Appendix A for how to generate such diagrams). |
119 |
|
The dark, orthogonal square is a single pixel, enlarged, over which the contributions |
120 |
|
from texels around it are shown by their color. |
121 |
|
a) bilinear, b), c) trilinear filtering. |
122 |
The trilinear footprint is the weighted sum of two bilinear footprints. |
The trilinear footprint is the weighted sum of two bilinear footprints. |
123 |
In c), the texture is mapped quite anisotropically and the |
In c), the texture is mapped quite anisotropically and the |
124 |
blurring effect of trilinear filtering is obvious - the footprint |
blurring effect of trilinear filtering is obvious - the footprint |
125 |
is much larger in the XXX direction than it should be. |
is much larger in the XXX direction than it should be. |
126 |
These filters were probed on an XXX. |
Hardware anisotropic filtering. |
127 |
The diagrams assume a box filter for generating the mipmaps, |
On the same card and same texture coordinates as |
128 |
as contributions from different mipmaps are directly blended |
c), but with anisotropic |
129 |
over each other. |
filtering enabled, the PFSS diagram shows a much better |
130 |
|
footprint fitting more closely around the pixel. |
131 |
} |
} |
132 |
\end{figure} |
\end{figure*} |
133 |
|
|
134 |
|
|
135 |
% \subsection{Anisotropic texture filtering} |
% \subsection{Anisotropic texture filtering} |
136 |
|
|
137 |
- basic anisotropic solution: more samples from the mipmaps than the 8 used for |
- basic anisotropic solution: more samples from the mipmaps than the 8 used for |
138 |
trilinear - better approximation of EWA. Modern graphics cards support up to XXX samples |
trilinear - better approximation of EWA. Modern graphics cards support up to XXX samples |
139 |
|
XXX Feline texram talisman |
140 |
|
|
141 |
- Unextended OpenGL aniso: article on using lod bias etc to get it\cite{olano01vertexbasedaniso} |
- Unextended OpenGL aniso: article on using lod bias etc to get it\cite{olano01vertexbasedaniso} |
142 |
|
|
143 |
- supersampling: FSAA / as above; however, most cards focus on multisampling, not supersampling - |
- supersampling: FSAA / as above; however, most cards focus on multisampling, not supersampling - |
144 |
no help for textures |
no help for textures |
145 |
|
|
|
- Graphics companies unfortunately do not provide ... |
|
|
|
|
|
|
|
|
- In this article, we argue that isotropic situations should be explicitly avoided |
|
|
in 2D orthogonal rendering - better quality with aniso |
|
|
|
|
|
- For text, setting of the problem: orthogonal transformations are most important, |
|
|
TrueType shows maybe not the right model but ... |
|
|
|
|
|
|
|
|
- case we consider: sharp edges, orthogonal (or nearly so) transformations, e.g. text |
|
|
|
|
|
- for sharp edges / small features, even under orthogonal transformations, trilinear bad: |
|
|
sampling at too low a resolution for much of the time! |
|
|
|
|
|
- LOD bias sharpening causes spatial and temporal aliasing (flickering) |
|
|
|
|
|
- simple solution for improving the situation in one direction: |
|
|
stretch the texture in one direction, squish back by texture coordinates. activate |
|
|
the aniso filter. Aniso filters planned so that they don't flicker, either. |
|
|
|
|
|
|
|
|
\begin{figure} |
|
|
a)\\ |
|
|
b)\\ |
|
|
c)\\ |
|
|
\caption{ |
|
|
\label{figaniso} |
|
|
Hardware anisotropic filtering. |
|
|
On the same card and same texture coordinates as |
|
|
Fig.~\ref{figbitrilinear} c), but with anisotropic |
|
|
filtering enabled, the PFSS diagram shows a much better (smaller) |
|
|
footprint. |
|
|
} |
|
|
\end{figure} |
|
146 |
|
|
147 |
|
|
148 |
\section{Why does stretch and squish improve image quality?} |
\section{Why does stretch and squish improve image quality?} |
149 |
|
|
150 |
|
- quality of trilinear filtering result depends strongly on subpixel position |
151 |
|
|
152 |
\begin{figure} |
\begin{figure} |
153 |
a)\\ |
a)\\\includegraphics[width=\snapsize]{snapsps/aniso-gf4go-ortho-trilinear.ps}\\ |
154 |
b)\\ |
b)\\\includegraphics[width=\snapsize]{snapsps/aniso-gf4go-ortho-stretchsquish.ps}\\ |
155 |
\caption{ |
\caption{ |
156 |
\label{figstretchsquishsamples} |
\label{figstretchsquishsamples} |
157 |
PFSS diagrams of an orthonormal rendering situation, |
PFSS diagrams of an simple rendering situation, |
158 |
showing how stretch-squish works. |
showing how stretch-squish works. |
159 |
a) Normal trilinear filtering. Playing around with a view like this of filtering |
a) Normal trilinear filtering. |
|
can show how \emph{bad} trilinear filtering really is. |
|
160 |
b) Stretching the texture and squishing it allows more samples to be used |
b) Stretching the texture and squishing it allows more samples to be used |
161 |
through an anisotropic filter. The footprint in XXX direction is much closer |
when using an anisotropic filter. The footprint in XXX direction is much closer |
162 |
to the actual pixel; there is less blur in the output. |
to the actual pixel; there is less blur in the output. Here, 2x anisotropy was |
163 |
|
used; using more anisotropy sharpens the filter further. |
164 |
} |
} |
165 |
\end{figure} |
\end{figure} |
166 |
|
|
178 |
} |
} |
179 |
\end{table} |
\end{table} |
180 |
|
|
181 |
\begin{figure} |
\def\fontexamplesize{8cm} |
182 |
|
\begin{figure*} |
183 |
|
\includegraphics[width=\fontexamplesize]{snapsps/aniso-font-trilinear.ps} |
184 |
|
\includegraphics[width=\fontexamplesize]{snapsps/aniso-font-lodbias.ps} |
185 |
|
\includegraphics[width=\fontexamplesize]{snapsps/aniso-font-horiz2.ps} |
186 |
|
\includegraphics[width=\fontexamplesize]{snapsps/aniso-font-vert2.ps} |
187 |
\caption{ |
\caption{ |
188 |
\label{figexamples} |
\label{figexamples} |
189 |
Magnified examples of text filtered using several algorithms. |
Magnified examples of text filtered using several algorithms. |
190 |
} |
} |
191 |
\end{figure} |
\end{figure*} |
192 |
|
|
193 |
- comparison: |
- comparison: |
194 |
|
|
204 |
NV GF FX supports that, nothing else. Others can only bias the lod so have to use supersampling 1 or 2x2 |
NV GF FX supports that, nothing else. Others can only bias the lod so have to use supersampling 1 or 2x2 |
205 |
|
|
206 |
|
|
207 |
|
|
208 |
- benefit / cost ratio analysis: how much slower than trilinear and |
- benefit / cost ratio analysis: how much slower than trilinear and |
209 |
how must faster than real supersampling -- does aniso filter provide |
how must faster than real supersampling -- does aniso filter provide |
210 |
``optimal'' quality / cost ratio? |
``optimal'' quality / cost ratio? |
213 |
|
|
214 |
\section{Conclusion} |
\section{Conclusion} |
215 |
|
|
216 |
|
- In this article, we argue that isotropic situations should be explicitly avoided |
217 |
|
in 2D orthogonal rendering - better quality with aniso |
218 |
|
|
219 |
|
|
220 |
- described how hardware implementations of anisotropic filtering can be probed |
- described how hardware implementations of anisotropic filtering can be probed |
221 |
for better understanding |
for better understanding |
222 |
|
|
248 |
\appendix |
\appendix |
249 |
\section*{Appendix} |
\section*{Appendix} |
250 |
|
|
251 |
\section{Probing hardware anisotropic filters} |
\section{Probing hardware texture filters for drawing realistic PFSS snapshots} |
252 |
\label{secprobing} |
\label{secprobing} |
253 |
|
|
254 |
|
In our investigations for this article, we found the pixel footprint |
255 |
|
diagrams in screen space (PFSS) diagrams most useful for understanding |
256 |
|
the properties of a filtering method w.r.t.~anisotropy. |
257 |
|
|
258 |
In this Section, we |
In this Section, we |
259 |
|
- Graphics companies unfortunately do not provide ... |
260 |
|
|
261 |
|
The diagrams assume a box filter for generating the mipmaps, |
262 |
|
as contributions from different mipmaps are directly blended |
263 |
|
over each other. |
264 |
|
|
265 |
This seems to be a well-known technique that has not so far been published |
This seems to be a well-known technique that has not so far been published |
266 |
anywhere |
anywhere |
267 |
|
|
268 |
|
|
269 |
|
- Graphics companies unfortunately do not provide ... |
270 |
|
|
271 |
Digit-life XXX NVIDIA, ATI patterns |
Digit-life XXX NVIDIA, ATI patterns |
272 |
|
|
273 |
- for careful work, you'll want to know what your driver is doing |
- for careful work, you'll want to know what your driver is doing |