BlackHC's Adventures in the Dev World

Reading the Deep Learning Book - Chapter 2

Andreas 'BlackHC' Kirsch — Mon, 27 Mar 2017 23:48:34 +0000

This are my notes and observations from reading the Linear Algebra chapter of the Deep Learning book.

The following notes are presented in order of value to the reader. I start with a discussion of the Moore-Penrose pseudoinverse, followed by a short reflection on "broadcasting".

Moore-Penrose pseudoinverse

The Moore-Penrose pseudoinverse is defined as:

\[ A^{+}:=\lim _{\alpha \searrow 0}\left( A^{T}A+\alpha I\right) ^{-1}A^{T} \]

You can click on the formula to jump into the book. (It's really awesome that all of it is online and you can deep link to pages.)

This formula is mystifying. What is its origin? And why does it do what the book says it does?

We can get a feeling for its origin by looking at the equation $Ax=b$ that it solves:

\[ \begin{align*} Ax &=b &|& A^{T}\cdot \\ A^{T}Ax &=A^{T}b &|& \left( A^{T}A\right) ^{-1} \cdot \\ \left( A^{T}A\right) ^{-1}A^{T}Ax &=\left( A^{T}A\right) ^{-1}A^{T}b & & \\ x &=\left( A^{T}A\right) ^{-1}A^{T}b & & \end{align*} \]

So we start with the equation that we want to solve and find a more complex form for solving for $x$ than the usual left-multiplication with the inverse $A^{-1}$. This is already useful if $A^T A$ is invertible and $A$ is not square (because matrix inverses only exist for square matrices).

This already looks similar to the pseudo-inverse, except for the $+\alpha I$ term.

Aside: there is also a right pseudo-inverse

We can guess its form by starting with:

\[ \begin{align*} yA&=b&|&\cdot A^{T}\\ yAA^{T}&=bA^{T}&|&\cdot \left( AA^{T}\right) ^{-1}\\ yAA^{T}\left( AA^{T}\right) ^{-1}&=bA^{T}\left( AA^{T}\right) ^{-1}\\ y&=bA^{T}\left( AA^{T}\right) ^{-1}\end{align*} \]

From this, we can make an educated guess:

\[ ^{+}A = A^{T}\lim _{\alpha \searrow 0}\left( AA^{T}+\alpha I\right) ^{-1} \]

Since we cannot always invert $A^TA$ (respectively $AA^T$), an obvious question is:

Why can we invert $AA^T+ \alpha I$ for positive $\alpha$?

Let's examine this. For a matrix to be invertible, its kernel has to only contain the zero vector:

\[\ker \left( A^{T}A+\alpha I\right) =\left\{ 0\right\}\]

To prove that this is the case for $\left( AA^{T}+\alpha I\right)$, we need to show that:

\[ \left( A^{T}A+\alpha I\right) v = 0 \implies v = 0\]

So starting with the left side, we can rephrase it as follows:

\[ \begin{align*} & & \left( A^{T}A+\alpha I\right) v&= 0\\ & \Leftrightarrow &A^{T}Av+\alpha Iv&= 0\\ & \Leftrightarrow &A^{T}Av &= -\alpha v \end{align*} \]

If $v \ne 0$, this would mean that $-\alpha$ is a negative eigenvalue of $A^TA$ as $\alpha$ is $>0$.

Can $A^TA$ have negative eigenvalues?

Let's assume $v \ne 0$ and $-\alpha$ is a negative eigenvalue, that is $A^{T}Av = -\alpha v$ holds (and $\alpha > 0$). We can left-multiply with $v^T$ and obtain:

\[ \begin{align*} & \Leftrightarrow v^{T}A^{T}Av=v^{T}\left( -\alpha v\right) \\ & \Leftrightarrow \left\| Av\right\| _{2}^{2}=-\alpha v^{T}v=-\alpha \left\| v\right\| _{2}^{2}\\ & \Leftrightarrow -\alpha =\dfrac {\left\| Av\right\| _{2}^{2}} {\left\| v\right\| _{2}^{2}}\geq 0 \end{align*} \]

(We can divide by $\left\| v\right\| _{2}^{2}$ because we assume $v\ne0$.) Now this means, that $-\alpha$ has to be $>0$, so it is not a negative eigenvalue and a contradiction to our initial assumption $\alpha > 0$. In fact, we have just shown that $A^TA$ in general can only have non-negative eigenvalues. (We can show the same for $AA^T$ the same way.)

Because $A^TA$ cannot have negative eigenvalues, this means that $v$ cannot be $\ne 0$, so $v = 0$. This is what we wanted to show, and this means that the kernel only contains the zero vector, and $AA^T + \alpha I$ is invertible for $\alpha > 0$.

What if $A^T A$ was invertible?

For a moment, let's consider the case when $A^T A$ is invertible: Is the Moore-Penrose pseudoinverse then equal to $\left (A^T A \right)^{-1} A^T$?

Matrix inversion is continuous in the space of invertible matrices. You might remember the definition of continuity from school. A better definition of continuity is that it means being able to swap function and limit application:

A function $f$ is continuous on its domain iff for any convergent series $x_n \to x$ for which $f \left( x \right )$ is defined \[ \lim_{n \to \infty } f \left ( x_n \right ) = f \left ( \lim_{n \to \infty} x_n \right ) = f \left( x \right).\]

Assuming $A^T A$ is invertible and using the fact that matrix inversion is continuous for invertible matrices, we now see:

\[ \begin{align*} \lim _{\alpha \searrow 0}\left( A^{T} A +\alpha I\right) ^{-1} A^T &= \left( \lim _{\alpha \searrow 0} A^{T} A +\alpha I\right) ^{-1} A^T \\ &= \left( A^{T}A+0 \, I\right) ^{-1} A^T \\ &= \left( A^{T}A\right) ^{-1} A^T \end{align*} \]

So in this special case, the pseudoinverse is exactly the solution we have come up with ourselves.

Two properties of the pseudoinverse

There are two properties mentioned in text that are interesting but not obvious:

When $A$ has more columns than rows, then solving a linear equation using the pseudoinverse provides one of the many possible solutions. Specifically, it provides the solution $x=A^+y$ with minimal Euclidean norm $\left\| x \right\|_2$ among all possible solutions. When $A$ has more rows than columns, it is possible for there to be no solution. In this case, using the pseudoinverse gives us the $x$ for which $Ax$ is as close as possible to $y$ in terms of Euclidean norm $\left\| Ax - y \right\|_2$.

These properties are not obvious and their deduction is enlightening towards the chosen definition of the pseudoinverse, specifically the use of the limit and the constraint of $\alpha$ to be $>0$.

To prove the properties, let's look at the following regularized minimum-least-squares problem. This is something that is formulated in more detail in Chapter 4 of the book, but it is quite useful here:

\[\min _{x}\left| \left| Ax-b\right| \right| _{2}^{2}+\alpha \left\| x\right\| _{2}^{2}, \, \alpha > 0\]

Let's solve this optimization problem. First, we set up a cost function:

\[ \begin{align*} c_\alpha\left( x\right) &=\left\| Ax-b\right\| _{2}^{2}+\alpha \left\| x\right\| _{2}^{2}\\ & =\left( Ax-b\right) ^{T}\left( Ax-b\right) +\alpha x^{T}x\\ & =x^{T}AAx-2b^{T}Ax+b^{T}b+\alpha x^{T} x \end{align*} \\ \]

The first derivative of $c_\alpha$ is:

\[ \nabla c_\alpha\left( x\right) =2A^{T}Ax-2A^{T}b+2\alpha x\]

And the second derivative is:

\[ H_\alpha \left ( x \right ) = 2A^T A + 2\alpha I\]

$H_\alpha$ is positive definite, that is $v^T H_\alpha \left ( x \right ) v \ge 0$ for all $v$. Why? We have already seen that $A^T A$ only has non-negative eigenvalues, so it is positive semidefinite by definition and $\alpha I$ is trivially positive definite for $\alpha > 0$. The sum of the two is positive definite again. Thus $c$ is a strictly convex function. This is along to lines of the one-dimensional case: when the second derivative is $> 0$ everywhere, the function is strictly convex. Convex functions have a global minimum and, for strictly convex functions, this global minimum is unique. So we know there is only exactly one point that minimizes the cost function $c_\alpha$.

We can determine this global minimum $x^*_\alpha$ by solving $\nabla c_\alpha \left ( x \right ) = 0$:

\[ \begin{align*} & & \nabla c_\alpha\left( x\right) &= 0 \\ & \Leftrightarrow & A^{T}Ax-A^{T}b+ax&=0\\ & \Leftrightarrow &\left( A^{T}A+\alpha I\right) x&=A^{T}b\\ & \Leftrightarrow & x&=\left( A^{T}A+\alpha I\right) ^{-1}A^{T}b\end{align*} \]

This is exactly the definition of the pseudoinverse without the limit. So we have found:

\[ x^*_{\alpha} = \underset{x}{\operatorname{argmin}} \left| \left| Ax-b\right| \right| _{2}^{2}+\alpha \left\| x\right\| _{2}^{2} = \left( A^{T}A+\alpha I\right) ^{-1}A^{T}b \]

with $c_\alpha \left (x^*_\alpha \right ) \leq c_\alpha \left ( x \right )$ for all $x$, and $x^*_\alpha$ denotes the minimum point.

This expression is continuous in $\alpha$, so we can take the limit $\alpha \searrow 0$. Remember, we need the constraint of $\alpha > 0$ for $A^TA + \alpha I$ to be invertible, so we can only take the limit from above. Taking the limit, we obtain:

\[ x^*= \lim_{x \searrow 0} \left( A^{T}A+\alpha I\right) ^{-1}A^{T}b \]

with $c \left (x^* \right ) \leq c \left ( x \right )$ for all $x$ with

\[c \left ( x \right ) = \lim_{\alpha \searrow 0} \left\| Ax-b\right\| _{2}^{2}+\alpha \left\| x\right\| _{2}^{2} = \left\| Ax-b\right\| _{2}^{2}.\]

What do we gain from this? Well for one, we now know that this expression minimizes $\left| \left| Ax-b\right| \right| _{2}^{2}$. Furthermore, if there is a solution for $A x =b$, we can easily use a similar approach to see that the solution $x^*$ is smaller under Euclidean norm than any (other) solution $\hat x$ for $Ax=b$.

We do this in two steps. First, we observe that

\[A\hat x = b \Leftrightarrow A \hat x - b = 0 \Leftrightarrow \left\| A\widehat {x}-b\right\| _{2}^{2} = 0 \]

and, because $x^*_\alpha$ minimizes $c_\alpha$,

\[ \begin{align*} & & c_{\alpha}\left( x_{\alpha }^{*}\right) &\leq c_{\alpha }\left( \hat {x}\right) \\ & \Leftrightarrow & c_{\alpha}\left( x_{\alpha }^{*}\right) &\leq \left\| A\hat {x}-b\right\| _{2}^{2}+\left\| \hat {x}\right\| _{2}^{2}=0+\alpha \left\| \hat x\right\| _{2}^{2} \\ & \Leftrightarrow & c_{\alpha}\left( x_{\alpha }^{*}\right) & \leq \alpha \left\| \hat x\right\| _{2}^{2} \end{align*} \]

This expression is again continuous, so we can take the limit $\alpha \searrow 0$, and we see:

\[\begin{align*} & c \left( x^{*} \right) \leq 0 \\ \Leftrightarrow & \left\| Ax^{*}-b\right\| _{2}^{2} \le 0 \end{align*} \]

Because norms are always non-negative, we have $0 \le \left\| Ax^{*}-b\right\| _{2}^{2} \le 0$, so $\left\| Ax^{*}-b\right\| _{2}^{2} = 0$. And we have observed above that this is equivalent to $Ax^{*} = b$. So if there is at least one exact solution to the problem, we are sure to obtain an exact one, too. To be fair, we could have deduced this in the previous section. However, we can take another limit on the inequality $c_{\alpha}\left( x_{\alpha }^{*}\right) \leq \alpha \left\| \hat x\right\| _{2}^{2}$ and obtain a more interesting result. This time, we only take the limit $\alpha \searrow 0$ of $x^{*}_\alpha$, but keep $c_\alpha$ fixed:

\[ \begin{align*} &&c_{\alpha}\left( x_{\alpha }^{*}\right) &\leq \alpha \left\| \hat x\right\| _{2}^{2} \\ \Rightarrow && c_{\alpha}\left( \lim_{\alpha \searrow 0} x_{\alpha }^{*}\right) &\leq \alpha \left\| \hat x\right\| _{2}^{2} \\ \Leftrightarrow && c_{\alpha}\left( x^{*} \right) &\leq \alpha \left\| \hat x\right\| _{2}^{2} \\ \Leftrightarrow && \left\| Ax^{*}-b\right\| _{2}^{2}+\alpha \left\| x^{*}\right\| _{2}^{2} &\leq \alpha \left\| \hat x\right\| _{2}^{2} \\ \Rightarrow && \alpha \left\| x^{*}\right\| _{2}^{2} &\leq \alpha \left\| \hat x\right\| _{2}^{2} \\ \Leftrightarrow && \left\| x^{*}\right\| _{2}^{2} &\leq \left\| \hat x\right\| _{2}^{2} \end{align*} \]

Here, we use $\left\| Ax^{*}-b\right\| _{2}^{2} \ge 0$ to be able to drop the term and $\alpha > 0$ to preserve the direction of the inequality, and we obtain the second property about the length of $\left \| x^{*} \right \|$.

Now we have proved both properties that were mentioned in the book. On the one hand, if there are solutions, we will obtain one with minimal norm. On the other hand, if there are no solutions, using the pseudoinverse will provide us with an $x$ that at least minimizes $\left| \left| Ax-b\right| \right| _{2}^{2}$.

To convince yourself that the expressions above are indeed continuous, we can use the technical argument that we can rewrite $c_\alpha \left (x \right )$ into $c \left (\alpha, x \right )$ and see that it is a continuous function in two variables instead of going the route of using functional analysis and treating $c_\alpha$ as a convergent series of functions.

Broadcasting notation: oh my...

The broadcasting notation in the Deep Learning book is weird. For one moment, let's ignore its origins from numpy, and let's look at what's happening.

In the context of deep learning, we also use some less conventional notation. We allow the addition of matrix and a vector, yielding another matrix: $C=A+b$, where $C_{i,j}=A_{i,j}+b_j$. In other words, the vector $b$ is added to each row of the matrix. This shorthand eliminates the need to deﬁne a matrix with $b$ copied into each row before doing the addition. This implicit copying of $b$ to many locations is called broadcasting.

Let's say $A \in \mathbb{R}^{3 \times 3}$ and $b \in \mathbb{R}^3$, for example:

\[ A = \left[\begin{matrix}0 & 0 & 0\\1 & 1 & 1\\2 & 2 & 2\end{matrix}\right] , b = \left[\begin{matrix}1\\2\\3\end{matrix}\right] \]

Then, with broadcasting:

\[ A + b = \left[\begin{matrix}1 & 2 & 3\\2 & 3 & 4\\3 & 4 & 5\end{matrix}\right] \]

How do we get there? Essentially, we take the vector $b \in \mathbb{R}^3$, interpret it as a row vector, and add it to every row of the matrix, so:

\[ A + b= A + \left[\begin{matrix}1\\1\\1\end{matrix}\right] b^T \]

Wouldn't it make more sense to write this as $A + b^T$? The reason for the unintuitive notation lies in the details of broadcasting: whereas in maths, a vector $\mathbb{R}^3$ is identified as a column matrix $\mathbb{R}^{3 \times 1}$ , in the context of broadcasting it is treated as a row matrix $\mathbb{R}^{1 \times 3}$. This row matrix is then repeated across the $1$ dimension to make it into a $3 \times 3$ matrix that can be applied to $A$. However, for matrix multiplications, $b$ continues to be treated as column matrix. This is really confusing. So what's the reason for this ambiguity?

My best guess is that in ML contexts, data entries (examples) are often treated as row vectors. For example, a design matrix stores a number of examples, each in a separate row. So if you want to change the bias of your examples, you want to add a bias vector as row-vector to each row. Indeed, in chapter 8, broadcasting is used to express batch normalization in eq (8.35):

\[H'=\dfrac{H-\mu}{\sigma}\]

I don't think this excuses the lack of clarity introduced with this notation, but then again it is too late to change and make everybody use a transposed design matrix... :)

Looking back and ahead

There is much more I could write about, but I think these were the most interesting bits from my notes. We have revisited convex functions, continuity and limits to motivate the curious definition of the Moore-Penrose pesudoinverse. Last but not least, we have looked at the origin of the broadcasting notation and why it might be confusing to someone who is new to ML. Next up: reading Chapter 3 and thinking about probability theory.

Stay tuned,
Andreas

PS: Thanks to my friend Armin Krupp for suggestions and corrections of the draft. All current mistakes and factual errors have been added later.

Recent Medium Posts

Andreas 'BlackHC' Kirsch — Mon, 13 Mar 2017 21:48:34 +0000

I have published a couple of posts on Medium to see how it works:

A Dart REPL PoC - Hacking with Dart

A well-received post about https://github.com/BlackHC/dart_repl which is a PoC interactive shell for Dart. I hope I've been able to restore some Karma points with the Dart team over this #butrisesagainharderandstronger..

Metamath - Formalizing math

A small write-up about my impressions from learning about Metamath and some questions that I really, really want to find time to look into.

A shoddy implementation in Dart can be found here: https://github.com/BlackHC/dart_metamath_naive

Reading the Deep Learning Book - Chapter 1

As part of a long study break, I've started reading the http://www.deeplearningbook.org/. I will try to write down my thoughts for each chapter over time as a way of engaging with the material.

Stay tuned for more posts and also cross-overs.

I'm still trying to decide whether to post the more mathy stuff here.

Cheers,
Andreas

Markdown Blog

Andreas 'BlackHC' Kirsch — Sun, 18 Dec 2016 21:48:34 +0000

It's been a while, but now my blog is up using a static HTML generator with Markdown as underlying format \o/ The static HTML is generated using a couple of small go scripts that wrap pandoc and go's text/template. Converting all of my WordPress posts to MarkDown was as fun as such a task can be, and quite tedious. However, it was still nice to read everything I have ever posted again and allowed me to take a more holistic view on everything I've done. It has definitely helped me with some life decisions this year.

I don't think this is the end of my efforts to improve this blog because I really should move off the go scripts at some point, and the content menu is very mobile-first (it only looks and works well on my phone screen). Moreover, source highlighting does not work either.

The biggest positive take-away, apart from finishing this mini-project, was that committing all intermediate outputs to a repository during the conversion really helped with moving fast. Every time I changed something in the scripts or in the Markdown code, I'd see how it trickles down. With disciplined reviewing, this allowed me to avoid random breakages, and it made debugging quite easy and cheap because SmartGit's differ would do most of the work for me.

I used the opportunity to learn and write some simple Go code. I am split on the language. It feels crude and lacking abstractions. Also, ~~duck typing~~ structural typing makes it harder to understand and explore code. On the other hand, I have been told all of this is intentional to keep things simple. Indeed, it was always straight-forward to get the code to do the things I want, with few surprises, even though it did usually not feel particularly elegant.

My time estimate for the conversion is that I've spent about 1 week for research, 1 week to convert the posts and 1 week to learn go and write the scripts. All of this was spread out over about 2 years with bursts of work now and then.

I'm not sure what the next steps are for this blog. It feels good to have it looked down in HTML and Markdown in a clean version that is easier to back up. I want to play around with other ways of publishing and organizing information. This blog can be a way to centralize all my efforts and keep pointers and write-ups.

Stay tuned! :-)

Writer's block

Andreas 'BlackHC' Kirsch — Mon, 13 Jul 2015 21:48:34 +0000

I don't know what to write... to write is to create though. So let's create:

I've been busy for the last 2 years. Busy with work and busy living. Life always takes you to different places, and for me this was Zurich (from Munich) and Google (from university). Both changes were not that big and yet quite big. They have taken me far away from where I imagined myself to be in this moment of time 2.5 years ago. I was at a loss of words about this for long, but I think I'm finding my voice again. My life at Google is worth a series of articles at a later time, and Zurich is a whole different story with very little Computer Science in my spare time.

So let's go back to the topic: writer's block. How to overcome it? In a way, I have run out of creative steam during my master thesis, and even though I am busy at Google, it's a different kind of business to work in a big company and be told what to do. Definitely more boring than university usually :) In a way, my writer's block, which has kept this blog on hold for the last 24 months, has also been a creator's block. I have resolved to change this with a willingness to forego some sleep in the months to come and show more discipline and restraint in my outgoing nature ;)

My writer's block in the last year is also a consequence of my reluctance to produce content for the WordPress platform I'm using, which has proven quite painful to maintain over time. The plugins and hand-written glue that keeps some of the blog together is not holding as well as it used to anymore, and I'm not able to fix all the issues in my spare time.

It's more fun to create than to maintain in general.

In this spirit, I want to come up with a new blog platform to run this blog on. Assuming that everything around is not fitting my peculiar taste, I will have to come up with something new from old parts. A Frankensteinian blogging system. My main idea is to actually use unit-testing and regression testing on the components and content of the blog to identify issues and maintain a consistent quality overall. In particular, content should stay static and not break in unexpected ways when adding new content that is intended to be orthogonal.

Imitating PBRT-style literate programming in LaTeX

Andreas 'BlackHC' Kirsch — Sun, 24 Mar 2013 23:19:44 +0000

Today, I want to release another bit of code from my master's thesis. However, this time it won't be C++ code, instead I'm going to release some LaTeX code which I used to display source code fragments with.

Specifically the results look like this:

You can download the accompanying example PDF here.

This mirrors the style of PBRT, which uses literate programming to develop and explain an advanced raytracer. Only it doesn't require you to write source code that adheres to literal programming principles. I used it to take parts of the source code of my master's thesis and explain them.

The following LaTeX code is used to create the fragment above:

\defTaggedCurrentFileFragment<Another file fragment;recursive>+=

You can either inline code fragment using a special listing environment, or you can reference code fragments in an external file.

//<Another file fragment;recursive> =
int fac_r( int n ) {
    if( n <= 1 ) {
        return 1;
    }
    return n * fac_r( n - 1 );
}

Cross-referencing is allowed as well and forward page references are created just like in PBRT. However, I have not figured out how to create backward references.

You can find the LaTeX code on GitHub. I have not wrapped it into a LaTeX package because I don't consider it mature enough, but maybe someone finds it useful, and it eventually ends up in a package (that would be neat :)).

To close, I want to say that the book More Math into LaTeX ¹ has been very useful while creating these LaTeX macros. It is the first book that covers creating custom macros and commands understandable and in-depth. It feels good to finally understand a bit more about how you can customize LaTeX to fit your needs :)

Cheers,
Andreas

See also http://www.math.cornell.edu/~whieldon/Math224_files/More_Math_into_LaTex.pdf.↩

Creative learning, Scratch and alternatives for learning how to create games

Andreas 'BlackHC' Kirsch — Thu, 28 Feb 2013 22:02:19 +0000

I'm still working at university as a student researcher for the chair of software engineering in a very unconventional placement: I help prepare and teach courses about topics in computer science to pupils, usually age 12-16.

My university¹ actually invests a lot of resources in promoting engineering degrees and applied sciences because the schools cannot. Schools here have only so many resources, and the computer science and mathematics curricula are mostly boring and uninteresting. For me, computer science is a lot about being creative and about identifying and solving problems. I'm not sure I would have been interested in computer science that much if it had been just another subject like biology or geography.

The courses we give are usually just one or two hours long for groups that visit our faculty, but we also have courses that last a whole day or two and cover topics in more detail. We usually choose topics that are more interesting than what you might learn at school and that show that computer science is not just about typing in code for hours without end but consists of analyzing real-world problems, devising algorithms and tinkering.

Most pupils already know a bit about computer science, and some even know how to code a bit with Java, but they are always positively surprised when we work out together how to get out of a labyrinth², how to come up with sorting algorithms, or how to encrypt messages easily with some binary magic ³.

Right now I am tasked with updating and recreating a two-day course about creating games that was held two times in the last years. The original course was only intended for girls with no former coding experience and used Scratch as development tool.

The updated version should be a bit more advanced and introduce more elements of game development properly to be both fun and educational. For this, I examined Scratch and alternatives to determine how to proceed. It ended up consuming a whole day and when I was done I had a beautiful mind map covering quite a number of different tools and frameworks. Having some spare time, I decided to write it all up in a blog post :)

Scratch

Scratch is a programming language with an IDE that is targeted at young people who have never coded before. It has been developed by the Lifelong Kindergarten group at the MIT Media Lab.

Scratch

Coincidentally, I'm participating in an online lecture they are offering right now: Learning creative learning. It is free and quite interesting---and Scratch is mentioned and discussed there as well.

Scratch's user interface is very simple and intuitive to use. You can code programs without having to type code. Everything is based on blocks that can be dragged and dropped to create event-based behaviors. Users can draw their own sprites and animate them or easily import their own pictures and use them. Scratch has an active online community and it is easy to share your projects with others and get feedback, which encourages participation.

However, a more experienced user quickly finds many limitations; for example, it is not possible:

to create procedures, ie custom blocks, and the message sending blocks are too primitive to create really complex behaviors;
to rename variables, which makes refactoring impossible;
to change the sequence of statements easily; or
to create new sprites (game objects) dynamically.

Moreover, I already found a Heisenbug: I could not write a simple lock to keep an event script from being run multiple times that works at run-time---it only works as described in the docs during step debugging (at reduced script execution speed).

Nonetheless, I consider it a very valuable tool that can be used to ease children into coding. The current 1.4 release needs to be installed on your computer, but they are working on a 2.0 release that will run entirely in the browser. You can give its beta a try here.

Creating programs using drag and drop is a very nice idea and I really appreciate Scratch's user interface. And I'm not the only one: Scratch has spawned many spin-offs that expand on its design. This has been possible because its source code has been released. You can find it here.

I have also examined some Scratch spin-offs that I have found:

Snap

Snap (formerly called BYOB for Build Your Own Block) extends Scratch and adds support for custom blocks, first-class lists and procedures, and continuations. It becomes a lot more like a functional language this way. You can run it here. The available documentation explains the new concepts very well. A German translation is available for download, too. The last release is from 2011, so it might not be actively maintained anymore.

snap

StarLogo TNG

StarLogo TNG extends Scratch into 3D and has been developed at the MIT as well. It is targeted at multi-agent systems and older users. It is more complex and there are some interesting tutorials and workshops available online: one shows how to build an ant population that knows how to pick up objects; and another one shows how to develop an epidemic model.

MIT App Inventor

MIT's App Inventor lets you build mobile apps easily using a Scratch-based programming interface. It supports a WYSIWYG UI editor as well. This used to be Google App Inventor apparently.

GameFroot

GameFroot's editor looks pretty polished. It is browser-based, and it is very simple to create a side-scrolling platformer out of the box. However, scripts are not that important and most behavior is predefined. So it's probably not a good tool to tinker with. The games that have been created with it naturally look very similar.

GameFroot

Stencyl

Stencyl is another game-creating tool, but it looks very mature and there is a pro version that is sold for 80-150 USD per year to publish your games to mobile devices and computers. The normal version only supports browser games. It is updated regularly but there is no localization support (yet).

Stencyl

The documentation is fairly complete, and it supports writing behavior scripts by using a Scratch-like interface, which supports custom procedures as well, or by writing JavaScript code.

Lots of different games have been created with Stencyl. Here are some which I've tried and which are really fun to play:

The Little Who
Balls in Space
The Wish

I've also looked at IDEs that are not based on Scratch.

Etoys

Etoys is similar to Scratch but not as polished. However, it has been translated to many different languages. It also lacks the social community features which are so prominent in Scratch. Moreover, you cannot run projects in the browser straight-away because a special plugin has to be installed first. This all probably hinders its adaption.

When you actually give it a go, you will be surprised by how nice it is. It feels very different to Scratch because it uses a unified workspace. There is no special script plane like in Scratch. Everything is part of the workspace. For example, when you look through the tutorials, you not only see an elastic ball bouncing, but you also see the special workspace frame that contains all different animation frames and the script code to assign them to the ball one after the other to emulate its deformation. This way you don't have to imagine anything. It's all there in the open for you to see---and debug if necessary.

Sadly, there is not much documentation and for me it is not as intuitive as Scratch.

Alice

Alice is is similar to Scratch and Etoys, only that Alice is using 3D scenes and is geared towards story telling. It's quite to big with over 600MB, but for this you get quality models to play around with. I can't say much about its features. There is not much documentation available online. You probably need to buy their book "Learning to Program with Alice" to really learn more about it. I do not like this. However, there is some free stuff available. It is somewhat hidden, but here is a link.

Again, you only code using drag and drop, but you can switch to a Java mode which makes the blocks look like their equivalent Java statements.

Greenfoot

Greenfoot can be used to teach programming with Java in a game-oriented context. Programming is done in a normal text editor. Greenfoot displays a class hierarchy and allows you to place instances of actors in a 2D scene. You can then control them manually or subclass actors and override their methods.

It's fairly basic but probably a good introduction to Java programming. If you start out with Scratch or Alice, it should be a small step from drag and drop coding to "real" coding.

Microsoft SmallBasic

I have learnt programming using good ol' QBASIC on my parent's 386, so I wouldn't disagree with a Basic dialect as a first language. Microsoft gave it some love, and it looks polished. I'm not sure how many are using it because there is no social community linked to it and the last release is from 2011. There is no specific support for creating games and the language is quite limited.

Microsoft TouchDevelop

This is really cool. Microsoft Research has a developed a fully fledged script language for mobile devices, so you can code using a touch-based interface. You can read more about it in the free online book "touchdevelop - programming on the go". Of course, it also runs in your browser and you can try it yourself :)

I don't think it is the best choice for creating games, but you can certainly learn how to program with it by creating "serious" apps for your mobile device. Maybe this makes it interesting for adult beginners to programming.

That's it for today.

Cheers,
Andreas

Technische Universität München, http://www.in.tum.de ↩
Maze solving algorithm ↩
Xor cypher ↩

Al et WML

Andreas 'BlackHC' Kirsch — Tue, 26 Feb 2013 01:45:10 +0000

First, I have created some new pages regarding old university projects. Among them is a condensed page about light propagation volumes, which also made me update the project files to Visual Studio 2012, a page about my bachelor thesis in mathematics (Discrete Elastic Rods) and a page about my master thesis in computer science (Assisted Object Placement). I have not written about the latter two subjects before. Maybe I'll talk some more about them later and write a full post-mortem on them.

This is the first of a number of posts that will be related to my master thesis, or rather code drops from its code base. I have written about 60k LoC in the 6 months of my master thesis and there are a few bits that might be useful in the future.

The first one that I want to talk about is a very simple file format I came up with. Devising new text file formats is not something that I have been very keen about lately. Especially not as some many already exist. However, I have found none which has really fit my requirements:

minimal clutter (preferably indentation-based),
support for raw text inclusion, and
good C++ support.

JSON has too much clutter and doesn't support raw text. YAML, on the other hand, sounds like the perfect choice, even though it's not that easy to find a good library for it. However, when it comes to raw text, you run into the issue that tab characters are never allowed as indentation. Moreover, I was not very happy with the API choices and some bugs in the libraries that I tried to use.

So I decided to develop a very simple text-based data storage format:

WML - Whitespace Markup Language

I'll just start with an example, ie my readme. You can find the full readme here.

'Whitespace Markup Language':

    Aims::
        * very simple
        * no clutter while writing
        * only indentation counts
        * empty lines have no meaning
        * embedding text is easy
        * everything is a map internally


    Example:

        title       "test\t\t1"
        path        'c:\unescaped.txt'
        version     1

        content::
            unformated text

            newlines count here

        properties:
            time-changed    10:47am
            flags   archive system hidden

        streams:
            stream:
                data::
                    some data

                    this is nested too
                flags:
                    read
                    write
                    execute

            stream:
                data::
                    key names
                    dont have to
                    be unique (see stream)
                flags:
                    read
                    write:
                        users   andreas root

As you can see, it is a whitespace-based format inspired by Python. Like in Python, indentation is used to convey structure. A WML file represents a map structure: every node has a name and possibly multiple children. A node definition in a WML file either contains the node name first and then multiple children names (which won't have any children themselves), the node name followed by one colon to signify that a nested definition follows (similar to JSON), or the node name followed by two colons to signify a raw text block. Two different kinds of strings are supported: single-quoted raw strings that do not interpret escape sequences, and double-quoted C strings that support escape sequences.

All in all, the grammar is very simple:

Grammar::
    INDENT, DEINDENT are virtual tokens that control the indentation level
    NEWLINE is a line break

    Indentation is done with tabs only at the moment.

    Here is a rough EBNF syntax for WML:

    root: map

    value: identifier | unescaped_string | escaped_string

    identifier: (!whitespace)+
    unescaped_string: '\'' (!'\'')* '\''
    escaped_string: '"' (!"\"")* '"' with support for \t, \n, \\, \', and \"

    key: value

    map: map_entry*

    map_entry: inline_entry | block_entry

    inline_entry: key value+ NEWLINE
    block_entry: key ':' ( ':' NEWLINE INDENT textblock DEINDENT | NEWLINE INDENT non-empty map DEINDENT )

Note::
    This file is itself a WML file and root["Whitespace Markup Language"]["Example"].data() is the example WML node

I've used this format for custom shaders as well as for my settings files and the declaration files of my test scenes:

boxes:
    -:
        name 'platform, brown'
        size 20 2 20
        webColor 945412
    -:
        name 'platform, green'
        size 20 2 20
        webColor 129429
    -:
        name 'platform, muddy blue'
        size 20 2 20
        webColor 6488a5
    -:
        name 'platform, light blue'
        size 20 2 20
        webColor e5eaf1

I've uploaded the current code for WML to GitHub, and you can find the code here. The API supports an overloaded index operator to access children of a node and contains both a parser and an emitter for WML.

Afterthoughts

First, the current API isn't brilliant. It would be nice to separate the data model from the parser and emitter by using templates and type traits to improve abstraction. I think I might go and investigate different API types in the future and see which one works best for some simple cases.

Second, it would be possible to reduce clutter even more and remove the need for single colons to denote nested maps. A even simpler format could look like this:

nodeA
    nodeB nodeC nodeD
        nodeE
        nodeF
    nodeG:
        raw data
            with fixed indentation

        would be interpreted as "raw data\n\twith fixed intendation\n..."

This would yield the following JSON-equivalent:

{ "nodeA": { "nodeB": {} , "nodeC": {} , "nodeD": {}, "nodeE": {}, "nodeF": {}, "nodeG": { "raw data..." : {} } } }

This still separates raw text from normal data. A node that contains raw text can never contain other children this way. However, I cannot think of a good way to accomplish that without introducing a special character to end a raw text block.

That's it for now :)

Unity Prototype Project

Andreas 'BlackHC' Kirsch — Wed, 06 Feb 2013 12:48:17 +0000

For the last two days I've tried out the Unity engine with a friend from university, Andreas Ostermaier. If you don't know the Unity engine, go and check it out.

Its design is very similar to Torque 2D. I can't tell who used it first, but Unity's design is more mature than what I remember from Torque 2D 1.3 back when I was still using it in 2007. A scene is made up of game objects. A game object is a container of components (also called behaviors)¹. A game object always contains a Transform component which places it in the scene (and in the scene hierarchy). Usually, it contains a Mesh component and a Physics component for handling rendering and collision detection. But sometimes empty game objects are useful as well: as respawn points for example. Custom scripts, written in JavaScript or C#, can be wrapped in a Script component and tied to game objects in the same way.

Templates/prototype objects², called prefabs, are used to avoid creating all game objects by hand. Instances of a prefab link back to the prefab, that is the original object, and automatically inherit changes to it (or the children in its hierarchy). Torque 2D 1.3 lacked this feature: GarageGames had an editor plugin in development when I was doing contract work for them but I'm not sure it has ever been released.

We have spent the last two days implementing a PoC for a game idea my friend came up with. The idea is about being able to switch between 2D and 3D in a platformer to solve puzzles. It is about reinterpreting a level when it is seen from a 2D perspective. For example, two separate tiles above an acid pool that cannot be reached by jumping from one to the other can suddenly be connected by switching to 2D because they look connected.

PoCDimRunner 3D PoCDimRunner 2D >

You can play the PoC build here, using Unity's webplayer, or download it here in a .zip archive. Use the left mouse button or 'c' to switch between 2D and 3D.

There are a few games that are similar to this idea: Echochrome, Super Paper Mario, and Crush (and its sequel Crush 3D).

You can find the code on GitHub (ie here).

Cheers,
Andreas

See eg here, here, and here for an overview.↩
See http://en.wikipedia.org/wiki/Prototype_pattern[[2]]↩

Presentation (seminar) about convex analysis

Andreas 'BlackHC' Kirsch — Sat, 29 Sep 2012 16:19:17 +0000

Last year (yes, I'm really slow with writing things up "lately") I did a presentation about convex analysis as part of a seminar about inverse problems in imaging.

I want to write about it today.

The seminar was based on the book 'Handbook of Mathematical Methods in Imaging (Springer Reference)'. It's very expensive and, in my opinion, pretty useless to learn about any details. At least the chapter about duality and convex programming was very dense and you could not learn anything from it without consulting other book.

The presentation was supposed to give an overview over duality and convex analysis and serve as introduction. Summarzing 42 very concise pages in 45 minutes is impossible, so I had to choose a few topics that give an overview over the important concepts and go into detail there.

The book itself does not really contain many proofs which makes it hard to follow. As a mathematician I like proofs. They help clear things up usually and they can give valuable insight into the methods of a theory when they're good.

But proofs suck when you see them on powerpoint slides. They also suck when you're the presenter.

So I gave a blackboard presentation---using a few slides only to show some pictures and sketches which I could not draw well enough on a blackboard.

To prepare the presentation I used OneNote and my Bamboo tablet. I scribbled all down over a weekend in one OneNote notebook and later extracted some sketches into another notebook that I used instead of a PowerPoint presentation. I exported the original notes as PDF and used this as handout after the presentation.

The presentation was alright. I had fun writing everything down on the blackboard and developing the subset of the theory I wanted to present, and the audience was content with it, too, because it was just at the right speed and like a lecture everybody was used, too.

In retrospect I can say that using OneNote and a tablet to create a blackboard presentation was the best decision. I don't want to think of the time I would have lost if I had created everything with LaTeX or PowerPoint.

Long story short: here are my notes :)

Cheers, Andreas

Molecular Dynamics and CUDA---my interdisciplinary project

Andreas 'BlackHC' Kirsch — Mon, 16 Apr 2012 12:49:10 +0000

You have to do a interdisciplinary project for your Master's degree at the Technische Universität München. I decided to do it at the Chair for Scientific Computing.

My topic was the Efficient Implementation of Multi-Center Potential Models in Molecular Dynamics with CUDA. For this, I have parallelized the molecule interaction calculations in the molecular dynamics simulator MarDyn with CUDA, optimized the code, and added support for more advanced potential models on the GPU.

What is molecular dynamics?

Molecular dynamics deals with the simulation of atoms and molecules. It estimates the interaction between a high number of molecules and atoms using force fields which are modeled using quantum mechanics equations.

Such simulations are important for many different areas in biology, chemistry and material sciences. For example you can examine the folding of proteins or the nanoscopic behavior of materials. The simulations are sped up using sophisticated approximations and algorithms.

Molecules only have strong force interactions with molecules that are nearby. One of the first approximations is to take into account these strong interactions only and ignore weaker long-distance ones. This space locality of the calculations leads to the Linked Cell-algorithm. It divides the simulation space into a grid of cells and only interactions between molecules inside each cell and with nearby cells are calculated.

Molecules are composed of atoms which interact using different physical principles. This is approximated by having different sites in a molecule that use different potential models. The most common and simplest potential model is the Lennard-Jones potential.

What is MarDyn?

MarDyn is a molecular dynamics simulator that has been developed at the Technische Universität München. See Martin Buchholz's thesis for more information.

Its code is rather ugly and it features some crazy design decisions. I have been told that the people who were initially developing it where learning how to code in C++ at the same time---and it shows :(

Previous work

When I started, there was a 'working' implementation of Lennard-Jones potentials for molecules with only one site on the GPU. It used OpenCL and was the result from somebody else's IDP. However, it was not very useful: the code was crammed into one function and one letter variable names were used through-out.

The main aims of the project

The idea was to port the previous OpenCL implementation to CUDA and continue from there to add support for multiple sites with different potential models.

Chronology of my work

Porting the code to CUDA was a straight-forward API change. However, I found the original code to be impossible to read, maintain, and optimize. The logic behind the code was not clear or well explained.

Consequently my supervisor and I decided that I would rewrite everything and optimize it from the beginning. This took longer than expected and while the resulting parallelism and the logic behind it were clear in the code, code complexity was an issue. The code consisted of three helper functions and two kernel functions in two files and when I went and integrated support for multiple sites and potential models, it became clear that the current design was not clear enough to endure further feature additions and lacked the flexibility for quick changes.

Treating the old implementation as a prototype and knowing about many possible traps and fallacies, I set out and rewrote everything again. This time the focus was on modularity and separation of concerns instead of performance. Code architecture was the most important thing this time around. I scaffolded the new version around the old code, which was working correctly and already optimized. I embedded it into the new design step by step. Afterwards I optimized the code.

What I've done

I've:

worked with CUDA 1.x and 2.x,
used both the driver and the runtime API,
implemented a template-based, very modular code design,
tried lots of optimizations, and
learnt how to read PTX to find work-arounds for compiler bugs.

Sadly some of the optimizations didn't improve the performance noticeably. I think the monolithic, all-in-one approach I've used is the main issue.

My code uses the driver API, because it would be easy to dump all the CUDA calls and replay them later for debugging purposes.

If I could iterate over the code one more time, I'd try to use many small kernels instead of only two rather large ones. Currently the kernels sequentially calculate all potential models for the molecule pair that are needed (for each site). This makes it difficult to measure performance improvements and isolate bottlenecks. And I suspect it's slower than it has to be.

I'd also create a sandbox application to test and develop the kernels instead of using MarDyn as starting point, because it is unnecessarily huge and it is not really needed to verify that the CUDA code works (or not).

Final paper & code

You can find the final paper that contains a description of my code and the performance results here, and my code here (I cannot include MarDyn's code because its source is closed, but at least I can share my code).

Alternatives

I just want to share two more links:

HOOMD-blue is a molecular dynamics framework, which uses CUDA, too, and the code design looks what I probably should have done :)
VMD is a molecular visualization application, which looks promising, too

Cheers,
Andreas

A Long Journey: Acceler8 and TBB

Andreas 'BlackHC' Kirsch — Wed, 16 Nov 2011 22:40:13 +0000

A month has passed with Intel's Acceler8 competition and it has finally come to an end. It's been a long way from implementing the first sequential algorithm to having a fully-fledged parallel version.

I have never worked with Intel's Threading Building Block library before and it was a nice opportunity to examine it, since it offered a better abstraction than OpenMP or pthreads.

The documentation is very good and you quickly learn how to work with the library. One of the caveats is that I didn't have to use a low-level synchronization construct once in the development and everything worked fine without any race conditions or similar . The parallel_* functions (eg parallel_for, parallel_reduce, and parallel_scan) together with icc's C++0x support (lambda functions) allowed for very concise code and little programming overhead.

The implementation builds on Kadane's algorithm for the two dimensional case using prefix sums. One implementation that gets across the basic idea can be found here. Mine is similar and I simply parallelized as much as possible.

As you can see the outer two loops iterate over a two-dimensional range that is pretty much an upper right triangle of the whole possible domain. For this I've implemented a custom range that allows for better load balancing. A range in TBB defines an iteration range of any kind and supports a split operation that is used internally by the task scheduler to distribute the range dynamically on multiple threads as it sees fit.

Last but not least I came up with a way to parallelize the 1D part of Kadane's algorithm that is being used by splitting the column range into linear subranges and merging the subsolutions into one solution, ie a classical divide and conquer approach.

Because it's the most abstract yet interesting part of our implementation, I'm going to go into more detail here. :-)

How can you find the maximum subarray of a 1D array, if you know the maximum subarray of the two "halves" (they don't have to be split evenly)? Well, you don't, you need more information.

We calculate the following information for each chunk:

maximum subarray that starts at the beginning of the chunk
maximum subarray that ends at the end of the chunk
total sum
maximum subarray

It's easy to figure out how to merge these values for two neighboring chunks into the values of the merged chunk. The merged maximum subarray that starts at the beginning of the merged chunk is either said value for the left chunk or the total sum of the left chunk + that value of the right chunk. You can figure out how it works for the maximum subarray that ends at the end of the merged chunk :-) The maximum subarray is just the biggest of all merged values or the left maximum subarray or right one.

Using this idea you can use a simple parallel_reduce to parallelize Kadane's algorithm.

Of course, there is some overhead but for the right problem sizes this will be faster than the sequential algorithm as always.

Two more take-aways:

Always try to use language features like templates or lambda expressions to reduce duplicate code or make the code more concise.
Write unit tests. I have used googletest which is a very small but very capable library, and it has spared me a lot of debugging trouble.

Cheers :-)

Reading Nonfiction Books Quickly

Andreas 'BlackHC' Kirsch — Wed, 02 Nov 2011 13:56:29 +0000

I like to read books and I also like to read nonfiction books and in particular nonfiction books that I do not agree with.

Speed Reading

For this I've decided to look into speed reading. It is an umbrella term for methods that increase your reading speed while keeping a satisfactory comprehension level. A usual reading speed is 200-300 wpm (words per minute). Speed reading advertises increasing your speed to > 600 wpm.

I've searched around for a bit and read lots of blog posts and articles and here are my favorites:

http://www.fourhourworkweek.com/blog/2009/07/30/speed-reading-and-accelerated-learning/ is a good introduction to the main techniques of speed-reading. It also includes some exercises and is a short read.

http://noahfleming.com/blog/speed-read-like-rain-man-75-increased-reading-speed-in-20-minutes describes the author's personal experience with fourhourworkweek's article.

There is also speed-reading software available. The idea is your eye movement is the main hindrance to reading really fast, so this software displays the text for you to read in groups of several words in the center of the screen. This way you can read everything at once without having to move your eyes at all. As crazy as it sounds, it works to a great degree.

http://www.spreeder.com/ is an online reader that works this way. Give it a try! You can make it display the introduction text at 600 wpm or 800 wpm and see how you'll understand - you'll be surprised!

'The Speed Reading Workbook' is what I've been using to practise it. It's okay written and has plenty of exercises and includes timing and evaluation sheets, which are really useful for measuring your progress.

A good summary of many concepts can be found in the PowerPoint Presentation 'Double your Reading Speed in 25 Minutes'.

About Reading in General

http://www.matthewcornell.org/blog/2006/2/26/how-to-read-a-lot-of-books-in-a-short-time.html is a good read and http://pne.people.si.umich.edu/PDF/howtoread.pdf has many good suggestions about how to increase your reading productivity.

Own Expierence

I've been exercising with the workbook now and then for the past weeks and I've become faster but this is still an ongoing process for me, so I'm going to blog about it later (or rather update this post).

For what it's worth: my reading speed was 300 wpm and now it is around 550 wpm :-)

Presentation Advice

Andreas 'BlackHC' Kirsch — Sun, 14 Aug 2011 21:13:25 +0000

So.. the summer term has passed and I'm wrapping things up and one of the final things I want to get out of my notebooks into this blog are some notes I took for feedback during my Hauptseminar in my maths studies.

Everybody had to do a 60 minutes presentation and the audience was asked to take notes about the presentation style to give feedback later (in addition to paying attention to the presentations).

There were 4 blackboards in the room and a beamer.

Here are my notes:

4 pages are the most you can do in 60 minutes (ie write down on the blackboard)
really think about your audience and what you want them to take away and focus on that in your presentation
practice your presentation a few times before your big day
if you use formal mathematical symbolism to express an idea, make sure all symbols and variables are properly defined/explained (for it to make sense!)
never tell people to "only ask questions at the end because it would disturb your presentation"!
focus on important concepts and ideas
motivate definitions and formulas - prefer an algorithmic view on things
try to use images and graphs to visualize ideas
if you can't avoid complex definitions and deductions, use a handout
make sure the handout doesn't contain superfluous information which will only confuse readers
don't just copy your handout onto the blackboard, if you hand it out at the beginning of the presentation (it's okay, of course, if you do it afterwards)
never clean all blackboards at the same time
avoid long breaks in your presentation
keep on talking all the time, even while cleaning the blackboard to keep people engaged
if the blackboards are in front of each other, always start writing on the one on the back
if you can use PowerPoint, try to use it for complex/boring formulas or for visualizations, which are impossible to draw by hand
on the other hand, make sure, you don't put things into PowerPoint, that would be better written out slowly on the blackboard
always listen to criticism politely, but don't take notes of it, while you're in front of your audience
don't degrade yourself by apologizing exaggeratedly for mishaps
put the sources of referenced material onto the slide the material is used in
don't waste time on complicated 'examples' that are too detailed to be of any use for the audience
don't let technicalities obscure examples - stick to the important parts of them
it's nice if the handout doesn't exactly match the presentation
always conclude your presentation with a summary of the main points of your talk
try to build intuition for your topic in your audience instead of explaining difficult proofs no one will remember or care for later anyway
make sure you have chosen the right priorities for your presentation and trim it down as much as possible

This was it :-)

Cheers,
Andreas

T61 Extractor

Andreas 'BlackHC' Kirsch — Sat, 13 Aug 2011 16:38:54 +0000

I've been a vivid fan of thesixtyone.com - until they changed the design. The old site can still be found here: http://old.thesixtyone.com/.

Many of my most favorite songs are from this site and I have only been able to listen to them through the site. However, since the aforementioned design change, the site has really died down in my opinion and I was fearful ever since that it would simply go down some day and vanish - taking my songs and playlists with it.

As counter-measure I've written a java console application to extract playlists and songs from thesixtyone. It uses Selenium to remote-control a FireFox instance that uses the normal user-interface to play songs and read in playlists.

I've uploaded the code to launchpad: https://launchpad.net/t61extractor/trunk I've tested and used it myself and it runs alright.

This was a small weekend project (or rather two weekend project) that I did a few months ago but I only found time now to write about it.

The code should be mostly self-explanatory and it's not a lot of code either.

Cheers,
Andreas

Books on Numerical Analysis

Andreas 'BlackHC' Kirsch — Mon, 01 Aug 2011 01:01:52 +0000

Due my writing an exam on numerical analysis I had the pleasure to look through lots and lots of books on numerical analysis, and here is a list of my favorite ones so far:

Afternotes on Numerical Analysis & Afternotes Goes to Graduate School by G. W. Stewart Both books are very readable and introduce many of the concepts in a light way that builds an intuitive understanding for them. It's possible to read the books cover to cover in a few days and you can learn a lot very quickly. They are also quite amusing:

"In the nineteenth century the Norwegian mathematician Niels Abel showed that no polynomial of degree five could be solved by a finite number of additions, multiplications, divisions, and root extractions. If we had a finite algorithm for finding eigenvalues of general matrices, we could apply it to companion matrices and make a fool out of Abel. Abel was no fool."
Numerical Linear Algebra by Trefethen Another very good book which I haven't used much personally, though
Numerical Methods by Kelley contains an okay introduction to CG and GMRES.
Numerische Mathematik I & Numerische Mathematik II are very good books, too. The first volume contains a good introduction to everything but ODEs and PDEs and the second volume consists of a very thorough overview of numerical algorithms for differential equations, including a nice introduction to the general theory of their solvability etc.
Numerical Analysis by Burden and Faires a very nice book that contains lots of visualizations and covers many topics

More to follow soon :-)

Cheers,
Andreas

ANTLR Stupidity (Warning 209)

Andreas 'BlackHC' Kirsch — Mon, 06 Sep 2010 11:16:48 +0000

I've been playing around with a Java grammar for ANTLR that was supposed to work straight-away but it did not, with very strange warnings and errors, that made it look like ANTLR only supports lexers with a lookahead of 1 character:

warning(209): ...: Multiple token rules can match input such as "'*'": STAR, STAREQ

while STAR matches only '' and STAREQ only matches '='. This is a huge w-t-f, especially if you have worked with ANTLR before and didn't have issues with this. This also contradicted all documentation you can find about ANTLR and its lexer rules.

I've spent a considerable amount of time with Google trying to find how to fix it. First I've found lots of posts on the ANTLR mailing list [antlr-interest] from people who had the same issue and no replies to them. People had issues with replacing character ranges with unicode ranges (or rather a huge list of unicode characters), which probably caused the problem in my grammar, too. Others found that ANTLR suddenly behaved as if it only had a one character lookahead, but only if more than 300 lexer rules were used in the grammar.

After searching for a long time and almost giving up on the mini-project I've wanted to use ANTLR for, I've found this post: http://www.antlr.org/pipermail/antlr-interest/2009-September/035954.html (which matches my problem more or less but with additional insight) and someone even replied (someone being the guy who maintains the C runtime of ANTLR): http://www.antlr.org/pipermail/antlr-interest/2009-September/035955.html

If you are sure that the messages are not correct and the lexer rules are not ambiguous, then you probably need to increase the conversion timeout:

-Xconversiontimeout 30000

if that does not work, then there is a conflict in your rules.
Jim

And that turns out to be the right advice and the remedy to my problems and the problems of lots of other people probably. However, no warning or error message I encountered mentioned that ANTLR's internal processing actually timed-out and there was no ambiguity in the grammar itself...

This comes to show that any good tool like ANTLR can quickly degrade to a piece of crap and a major source of annoyance if error and warning messages aren't clear and helpful.

On further investigation, you can trigger warnings that the conversion times out:

internal error: org.antlr.tool.Grammar.createLookaheadDFA(Grammar.java:1279):
    could not even do k=1 for decision 121; reason: timed out (>1ms)

but not consistently. I guess this is a bug - either in ANTLR or in ANTLRWorks... :-|

Limbo

Andreas 'BlackHC' Kirsch — Sun, 05 Sep 2010 13:51:35 +0000

Limbo is a game that has been released on Xbox Live Arcade some time ago, but I've only now found time to play it.

It is a very nice game - albeit a bit on the short side: it took me about 5 hours to finish it. But this is the only negative bit I can think of. Everything else about the game is very nice. It is very polished and it's a charm to play. You have to think about the puzzles for some time but the learning curve is okay and there were no unfair bits.

The game is black-and-white only (with shades of gray) and you play a boy that apparently got lost in a forest (this is how it starts) and you want to get out.<

The game mechanics are very simple: you can only move around, jump and hold on to things to drag them around (or press buttons, etc). In this regard the game is very similar to Another World (another very good game worth playing).

The puzzles are all phyiscs-based and because of this the aforementioned fairness is achieved: sometimes you just have to think a bit longer how to solve a puzzle but it is always logical.

The game is somewhat violent because it works using the "learning by dying" principle and if you don't enable the gore filter, you'll see the poor boy being halved, stabbed, squashed,... many times. However, since he is only a black shape, it's okay and won't put you off playing. The game uses many (and only) auto-saves to track your progress and you never feel punished for dying because you'll get another try straightaway without having to replay more than a few seconds.

"Learning by dying" works really well (and is also used in Another World): You don't have to worry about a "health bar" or rewind time or...

The difficulty of the game is just right and it gets harder as you progress through the world: you get to use more objects to solve the puzzles or will have to think of new ways to use physics to achieve your goals. However, one annoying bit is that also autosave points get moved further apart even when there is no need for it. This only happens in the last few "chapters" of the game though.

All in all it is very much a game worth playing and a very polished experience - kudos to the developers!

Light Propagation Volumes

Andreas 'BlackHC' Kirsch — Sun, 18 Jul 2010 15:44:39 +0000

I've finally finished my lab course last week - thanks to my supervisor Matthäus G. Chajdas - you can read his blog here -, it wasn't your usual lab course with work sheets and boring homework, instead I've been allowed to implement a nice paper about a Global Illumination approximation algorithm called (Cascaded) Light Propagation Volumes. It's been developed by Crytek and you can find more information (including some presentations and videos) on their server. (Note: this is an implementation of the I3D paper, not the earlier SIGGRAPH one.)

Sponza scene (direct + indirect lighting w/ occlusion)

Sponza scene (only indirect lighting w/ occlusion)

Sponza scene (ony direct lighting)

Sponza scene (boosted indirect lighting w/ occlusion)

Sponza scene (boosted indirect lighting w/o occlusion)

The algorithm approximates global illumination by rendering the light into a reflective shadow map, injecting it into a volume (using a spherical harmonics representation) and propagates the light flux in this volume (hence the name of the algorithm) and taking into account occlusion as possible extension.

The whole algorithm is physically motivated but corners everywhere, of course, to be more efficient. The paper also contains a few errors and doesn't explain everything needed to implement it in great detail (like eg the solid angles of the side faces), so I've written two documents detailing the mistakes I've found and the additional calculations I've performed.

You can find the mistakes here (including suggested corrections) and the full annotations document here.

Finally I've also uploaded the whole prototype (including my code licensed under the FreeBSD license and the media files) here - it's 68 MB big (and it's been compressed with 7zip with a compression mode that might not be supported by WinZIP. The Sponza model is from Crytek, too. You can download the original model and textures here. The project uses DirectX 10.1 and by default it won't run in DirectX 10, because it uses a texture format that is deprecated in D3D10 but supported again 10.1 (BGRA). See the comment by FatGarfield for the location that needs to be changed for it work in DX10, too. (However red and blue will be swapped then.)

Update: I've updated the project to support Visual Studio 2012. You can find the full download here (68 MB).

I haven't implemented cascaded LPVs and I also use only one light/RSM and only inject its depth into the occlusion volume, but the results already look very nice in my opinion.

Stay tuned for more :-)

Cheers,
Andreas

Panorama Stitching

Andreas 'BlackHC' Kirsch — Sat, 17 Jul 2010 11:00:04 +0000

I've finally come around to "clean-up" some old project I've had lying around for a few months and upload it. I'm talking about some Panorama Stitching code I wrote for our participation in Microsoft's Imagine Cup. I'm suppressing all memories of it since it was an epic fail, but at least I still have learnt quite a bit about computer vision and image processing - enough to know that it's incredibly hard to come up with stable algorithms and kudos to anyone working in the field.

Here is the code dump: PanoramaStitching.zip

It contains many small projects which usually use multiple pictures as inputs or multiple webcams (depending on code or chosen preprocessor macros).

The most advanced prototype is the SnapshotHomographyConfigurator, which allows you to determine homographies between multiple cameras at once by marking shared points between the images.

Another one which works okay is the PanoramaStitching project. It creates panoramas using spherical or cylindrical projections of the input images. However, it is very sensitive to translations of the viewpoint. It works quite well with optimal/artificial images:

(Note: the small misalignment on the right stems from moving the player position slightly. Usually you use a deghosting algorithm to remove such misalignments.)

I've used OpenCV for image processing and yaml for loading and storing settings (and also rapidxml). OpenCV's C++ wrapper is pretty awesome. It's not perfect but it makes life a lot easier.

Stay tuned for more code/project uploads soon :-)

PS: Here are the links to some papers which proved useful to me (I didn't implement most of them though, and some are implemented in OpenCV already):

Rotation of Low Order Spherical Harmonics

Andreas 'BlackHC' Kirsch — Sat, 19 Jun 2010 23:10:05 +0000

I'm currently working at university on implementing Light Propagation Volumes. The paper makes extensive use of spherical harmonics while the implementation uses the first two bands.

Below is a visualization of the first 4 bands of the SH basis functions (created using Mayavi):

The first 4 bands of the spherical harmonic basis functions

As you can see the first two bands are 4 functions, so 4 coefficients to store which conveniently fits into one RGBA texture.

One of the main transformations that is performed in the LPV paper is the rotation of the spherical harmonics representation of a clamped cosine lobe (that represents surface lighting) onto a normal vector direction. It took me a while to figure out, but actually it's quite easy, which is why I write about it :-)

The analytical presentation of the first four base functions is simple:

\[S_0 \left( x, y, z \right ) = \frac{1}{2 \sqrt{\pi}}\] \[S_1 \left( x, y, z \right ) = - \frac{\sqrt{3}}{2 \sqrt{\pi}} y\] \[S_2 \left( x, y, z \right ) = \frac{\sqrt{3}}{2 \sqrt{\pi}} z \] \[S_3 \left( x, y, z \right ) = - \frac{\sqrt{3}}{2 \sqrt{\pi}} x\]

To evaluate lighting with SH for some direction $v$, you first determine the coefficients/weights of the SH basis functions and then sum them up.

\[ L = \sum_i s_i \, S_i \left( v \right ) \]

Let's assume we know the coefficients $s^z_0, s^z_1, ...$ of the clamped cosine lobe around the z axis, then we can determine the lighting in direction v for the cosine lobe around the normal n by transforming it into the space where the normal coincides with the z axis (ie rotate n onto the z axis):

\[ L = \sum_i s^z_i \, S_i \left( R_{n \to z} \, v \right ) \]

where $R_{n \to z}$ is a rotation matrix that rotates n onto z.

The idea is to expand $S_i \left( R_{n \to z} \, v \right )$ and rewrite it in terms of $S_i \left ( v \right )$.

Before doing this, let's first take a look at the coefficients of the clamped cosine lobe:

\[\begin{align*} s^z_0 &=\frac{ \sqrt{ \pi } }{ 2 }\\ s^z_1 &= 0\\ s^z_2 &= \sqrt\frac{ \pi }{3}\\ s^z_3 &= 0\\ \end{align*}\]

The y and x direction are 0 because the cosine lobe is centered isotropic around the z axis:

So let's look at the expanded version of this formula if $r_1^T$, $r_2^T$, $r_3^T$ are the row vectors of the matrix, $v=\bigl(\begin{smallmatrix} x\\ y\\ z \end{smallmatrix}\bigr)$ and $R_{n \to z}=\left(\begin{smallmatrix} r_1^T\\ r_2^T\\ r_3^T \end{smallmatrix}\right )$, then:

\[ L = \sum_i s^z_i \, S_i \left( R_{n \to z} \, v \right ) = \sum_i s^z_i \, S_i \left( \left(\begin{smallmatrix} r_1^T \, v\\ r_2^T \, v\\ r_3^T \, v\end{smallmatrix}\right ) \right ) \] \[\begin{align*} L &= s^z_0 \, c_0\\ &+ s^z_1 \, (-c_1) \, r_2^T \, v \\ &+ s^z_2 \, c_1 \, r_3^T \, v\\ &+ s^z_3 \, (-c_1) \, r_1^T \, v \end{align*} \]

Since $s^z_1 = 0$ and $s^z_3 = 0$:

\[ L = s^z_0 \, c_0 + s^z_2 \, c_1 \, r_3^T \, v = s^z_0 \, c_0 + s^z_2 \, c_1 \, r_{31} \, x + s^z_2 \, c_1 \, r_{32} \, y + s^z_2 \, c_1 \, r_{33} \, z \]

\[ L = s^z_0 \, S_0 \left ( v \right ) - s^z_2 \, r_{32} \, S_1 \left ( v \right )+ s^z_2 \, r_{33} \, S_2 \left ( v \right ) - s^z_2 \, r_{31} \, S_3 \left ( v \right ) \]

Now the question is: what is the third row of $R_{n \to z}$? If we look at the inverse matrix instead: $R_{z \to n}$, we can immediately see that its third column has to be n, because \[ R_{z \to n} \, \bigl(\begin{smallmatrix} 0\\ 0\\ 1 \end{smallmatrix}\bigr) = n \] by construction. Since rotations are orthogonal matrices, the inverse is the same as the transposed, so we can deduce that the third row of $R_{n \to z}$ is the same as the third column of $R_{z \to n}$, that is: n. Thus with $n = \bigl(\begin{smallmatrix} n_x\\ n_y\\ n_z \end{smallmatrix}\bigr)$ we get:

\[ L = s^z_0 \, S_0 \left ( v \right ) - s^z_2 \, n_y \, S_1 \left ( v \right )+ s^z_2 \, n_z \, S_2 \left ( v \right ) - s^z_2 \, n_x \, S_3 \left ( v \right ) \]

So the SH coefficients of the clamped cosine lobe along n are:

\[ s^n_0 = s^z_0 = \frac{ \sqrt{ \pi } }{ 2 } \\ s^n_1 = - s^z_2 \, n_y = -\sqrt{ \frac{ \pi }{3} } \, n_y \\ s^n_2 = s^z_2 \, n_z = \sqrt{\frac{ \pi }{3} } \, n_z \\ s^n_1 = - s^z_2 \, n_x = - \sqrt{\frac{ \pi }{3}} \, n_x \]

This is it :-)

Cheers,
Andreas

PS: a few screenshots from the LPV project:

GPUPropCopy

noLPV_2

noLPV

LPV32P128C_2

LPV32P128C

Rigid Body Motion

Andreas 'BlackHC' Kirsch — Sun, 30 May 2010 15:10:11 +0000

Last week I had to give a presentation about Rigid Body Motion (ie the basics of rigid body physics and some general mechanics).

Here are two versions of my presentation (one with less text and one with more):

Rigid Body Motion Presentation as PPTX; Rigid Body Motion Presentation as PDF
Rigid Body Motion Full Version as PPTM; Rigid Body Motion Full Version as PDF

If you are truly interested in learning about rigid body physics, here are some books/links:

“Technische Mechanik 1 & 3” by Gross, Hauger, Schröder & Wall
“Physically Based Modeling” by Pixar (SIGGRAPH 2001)
“Mathematics for 3D Game Programming & Computer Graphics” by Eric Lengyel
“Physics for Game Developers” by David M. Bourg
“Game Physics” by David H. Eberly
Chris Hecker's Rigid Body Physics page

Cheers, Andreas

PowerPoint LaTeX

Andreas 'BlackHC' Kirsch — Sun, 30 May 2010 09:58:04 +0000

Hey everybody :-)

I only wanted to point out that I've uploaded a new and improved version of PowerPoint LaTeX at http://code.google.com/p/powerpointtools/ - it now supports MiKTeX /

I've also finally added a project page for it to this blog. More updates might follow soon if I find enough spare time :-)

Cheers,
Andreas

A quick note on quantifiers: For almost all and there exist infinitely many

Andreas 'BlackHC' Kirsch — Sat, 27 Mar 2010 14:55:00 +0000

It's been a while since my last post and now it's time for a mathematical post:

I'm currently preparing for a math exam (calculus) and I'm thinking it would be nice if there was a way to avoid much of the "for every $\epsilon > 0$ there is an $N \in \mathbb{N}$, so that for all $n \ge N$ some property ... holds" stuff you find in textbooks. Some textbooks actually shorten it to "for every $\epsilon > 0$ some property ... holds for almost all $n$".

However, I haven't found a quantifier to express this anywhere yet, so I'm proposing to introduce a new one:

$\tilde{\forall} x \in M: P(x)$ should mean "for almost all $x \in M$ $P(x)$ holds", which suggests that there are only finitely many elements for which it does not hold.

One can formalize this as:

\[ \tilde{\forall} x \in M: P(x) \equiv \exists n \in \mathbb{N}: \left | \left \{ x \in M \mid \neg P(x) \right \} \right | \le n \]

Of course, a new existential quantifier is required then, too (for negation):

$\tilde{\exists} x \in M: P(x)$ stands for "there exist infinitely many $x \in M$, for which $P(x)$ holds".

And this can be formalized as:

\[\tilde{\exists} x \in M: P(x) \equiv \forall n \in \mathbb{N}: \left | \left \{ x \in M \mid P(x) \right \} \right | > n \]

It's easy to see that \[\neg \left ( \tilde{\forall} x \in M: P(x) \right ) \equiv \tilde{\exists} x \in M: \neg P(x) \]

Thus one can use the two quantifiers just as one would use $\forall$ and $\exists$ usually. Note however than $\forall$ and $\tilde{\forall}$ don't interchange and neither do $\exists$ and $\tilde{\exists}$.

One last note: it might be worth using a different notation, for example: $\exists ^ \infty$ might be easier to understand than $\tilde{\exists}$, and $\forall ^ \approx$ might be better than $\tilde{\forall}$.

I'll try to formalize these quantifiers some more when I find some spare time.

Stay tuned for coding related updates soon :-)

Random Things

Andreas 'BlackHC' Kirsch — Fri, 27 Nov 2009 16:56:20 +0000

It's been a while since the last update. Here's a small update on what I'm thinking about various stuff.

Prototype

Gameplay is okay I guess. Nothing I would spent too much time on though. It looks a lot worse than GTA 4 though (same as Crackdown).

Resident Evil

Seems to be lots of fun and some nice graphics, too.

Red Faction: Guerrilla

Some weird texture filtering issues and the presentation doesn't knock off my chair (terrain popping and other ugliness), but multiplayer is loads of fun with friends. Physics isn't completely stable though. I played the game for one hour in MP with friends and there were quite a few cases where geometry dropped through the floor with enough pressure from above (after destroying a building).

Brüno

A bad and quite stupid movie (Ali G in Da House is probably the only movie I kinda like that stars Cohen). If you haven't watched it already, don't >_<

Shadow Complex

I bought shadow complex a few weeks ago and I have to say that it is an awesome game. I read somewhere that the developer used the Metroid series as inspiration and it shows. It's really fun to play and quite addictive. The graphics are pretty awesome (it is using the Unreal 3 engine) and the whole presentation is pretty polished. It certainly is worth its 1200 gamerpoints

Summing Formulas

A month ago I was doing some exercises in an analysis book (Königsberger) and found a nice/interesting problem:

Prove that if we denote of the sum of the numbers 1 to n to the p-th power by \[ S_n^p = 1^p + 2^p + ... + n^p \], then the following equation by Pascal holds: \[ (p+1) S_n^p + \binom{ p + 1 }{ 2 } S_n^{p-1} + \binom{ p + 1 }{ 3 } S_n^{p-2} + ... + S_n^0 = (n+1)^{p+1} - 1 \]

It can be used to get recursively/iteratively get formulas for sums of higher powers of numbers.

This is interesting because the proof doesn't need any advanced maths (like eg the Euler-MacLaurin formula, which can be used to show this, too).

You can download the proof and a few examples here.

Cheers

Extracting Information from StudiVZ

Andreas 'BlackHC' Kirsch — Mon, 19 Oct 2009 18:34:42 +0000

Some time ago somebody stole 1 million data records from StudiVZ, the German Facebook clone. I'm not exactly sure why people call the person a hacker who stole data, because it appears he simply wrote a tool that harvested the publicly available data from StudiVZ (which everyone with an account can view).

People on StudiVZ share all their data by default---contrary to Facebook which values a person's private data a lot more. Thus by simply opening each profile from a dummy user and processing the HTML data from StudiVZ one can extract a lot and some more information from random people who probably don't even know about it or don't care.. so I'm not sure about the stealing part.

Apparently there are some captcha's when you start browsing searches beyond a few pages. I guess that is where the hacking part comes in, because getting around a captcha probably constitutes hacking---maybe?

Anyway I think part of the media coverage is a bit ridiculous because anyone can write a simple harvester in an hour or two. It took me one and half hours, so I think I'm on the safe side with this estimate and I didn't really have a clue about this stuff before either.

Since I don't want to "hack", I've only written a very tame harvester. It connects to your personal StudiVZ account, and retrieves the name and profile ID (and thus profile URL) of all your friends in the "Meine Freunde" pages.

It could do a lot more with that like retrieving everybody's birthday or random pictures, but I'm too lazy to code that because you use the same pattern for extracting data over and over again and it stops being interesting quite fast.

You can download the project here. It is a one file C# project. I'm releasing it under GPL (whatever).

It's really easy to explain how it works:

It uses System.Net's HttpWebRequest and HttpWebResponse to get (and post) web pages.
StudiVZ (like every other portal) uses cookies, so I create a CookieContainer and use it in every http request.
There are a few hidden values that StudiVZ expects during login. I'm retrieving them from the main page using custom built regular expressions. I've found a handy AJAX tester for .NET regular expressions. It was really useful for building the expressions and debugging them. (BTW you can find all URLs I used in the comments.)
After login I use the same pattern: get page & parse using regex for everything.
Visual Studio has an awesome "HTML Visualizer" for strings. It displays the content of a string as HTML page, which is really nifty if you're doing anything related to HTML processing.

The code is quite ugly. Well, it's not production code and this is only meant as a proof of concept.

Also note that I have at most violated the AGB of StudiVZ and not committed any criminal acts and I'm not planning to sell my friend's profile IDs or data either :-)

Maybe someone can extend the code and make it more useful. I guess it would be fun to automatically download all your pictures (including tags) and feed them into flickr or picasa... but someone else can do that.

Cheers, Andreas

My Bachelor Thesis

Andreas 'BlackHC' Kirsch — Sat, 17 Oct 2009 13:43:19 +0000

Screenshot of Utah

For the last two months I have been working on my bachelor thesis at the Chair of Computer Graphics and Visualization. It is about "Multi-Tile Terrain Rendering with OGL/Equalizer". The chair has a very nice Direct3D 10 terrain rendering engine and they want to run it at the newly founded KAUST (King Abdullah University of Science and Technology) in a massive CAVE environment. A CAVE is a room whose walls are actually screens. The CAVE at KAUST even supports stereoscopic rendering, so in total 12 screens that you need to render to.

My job was to port a terrain engine from Direct3D to OpenGL and afterwards to the Equalizer framework, which is an open-source framework for parallelizing OpenGL applications.

You can find/download an online version of my bachelor thesis here. I'll upload the LaTeX at a later date and update this post.

I've spent the last months writing about all this, so I don't feel like talking about the thesis itself anymore. Instead the remainder of this post will contain a post-mortem of it.

Post-Mortem

Preparations

I think I did a good job at preparing myself for the thesis: I'm a LaTex noob, so I got 4 LaTeX books; and I'm a (paper-)writing noob, so I got 3 books about writing papers and writing good English.

The books have helped me a lot and it is always a good decision to get as many books as possible, because the more sources you have, the better. :)

I got pretty much all the LaTeX books from our university library:

[Einstieg in] LaTeX A big book (696 pages) but not particularly helpful. The reviewer on Amazon is pretty much right: it contains lots of information., but always stops short of providing the really useful details/information.
LaTeX - Ge-packt A small but thick pocket book (608 pages). I totally forgot I had it and it stood on my bookshelf until I found it yesterday. I've used it though while writing another paper last year, and it's pretty neat. It's a nifty reference, that's what it is.
LaTeX Hacks: Tipps und Techniken für professionellen Textsatz This book is very good. It contains 100 "LaTeX hacks" (on 416 pages). These are small, closed sections describing the solution to a specific problem you might encounter while using LaTeX. I've used the book a lot of times and it references many useful LaTeX packages, too---so it's a good start for searches.
Wissenschaftliche Arbeiten schreiben mit LATEX Get this book if you want to write your thesis with LaTeX. It's incredibly useful and covers most of what you'll need. It takes you on a journey through all features of LaTeX that you will use and describes the most common use cases. It even has a section describing how to fix common LaTeX errors like overflowed hboxes and others.

I've also bought some books about writing papers and the English language in general:

A Manual for Writers of Research Papers, Theses, and Dissertations, Seventh Edition: Chicago Style for Students and Researchers This book covers everything you need to know about the style and format of a thesis. It has a huge chapter about how to add citations and what format the bibligraphy should use, etc. In retrospect reading it was probably overkill, because a bachelor thesis apparently doesn't have very strict requirements (well, I'll see about that when I get my feedback..). It, however, contains a few nice chapters about editing your text and improving its readability, and the punctuation and spelling chapters are nice, too.
Oxford Guide to Effective Writing and Speaking I've only read this book in parts but it's not bad. It contains a useful introduction to English grammar, if you want to refresh that knowledge.
Oxford Guide to Plain English Get this book This book is simply awesome. Plain English refers to writing English in a simple and easily understood way. This book describes what makes a text 'plain'/easy to read and how to transform your texts to improve readability. It contains lots and lots of examples of complicated texts and shows how easy it is to improve them with few changes. The book itself is a very nice read and in parts even funny. The author clearly knows what he is teaching and it helped me a lot while writing my bachelor thesis.

Writing the Thesis

I had two months for the implementation of the project and for writing the thesis. The plan was to implement everything in August and write everything in September and be done at the beginning of October (before my birthday). So last week I ended up being in crunch mode working 15 hours on it every day :-|

The implementation went mostly as planned. I found a few driver bugs though, bought gDEBugger for debugging shader issues and found some bugs in it, too. I think I reported 5-7 bugs in gDEBugger's support forum in a matter of days and still have to write/extend a few reports.

I wrote my thesis in LaTeX and used the very good Texmaker as editor. It supports multiple file projects and allows you to select a main file (usually latex is called with the file that is currently being edited).

jEdit also supports LaTeX editing through a plugin: LaTeXTools. It isn't working as well as Texmaker though, but it generally has more text editing options and supports directory searches and regexp which is useful for a multi-file project.

I used subversion for version management, which proved useful at times, even though TortoiseSVN's difference viewer doesn't support line wrapping :(

For creating vector graphics, a very cool LaTeX package exists. It's called TikZ ("Tikz ist kein Zeichenprogramm"), and it allows you to create illustrations the LaTeX way: by coding them. See http://www.texample.net/tikz/examples/ for some neat looking examples.

I really wanted to use it to create my illustrations, but due to time constraints I couldn't read the whole manual (it's huge!) and even then it would have been too difficult. You see, it suffers from the same problem TeX and LaTeX suffer from in general. It's easy to write something that looks okay, but as soon as you want to tweak something to look exactly the way you imagine it and it is not a standard option, you end up reading weird package code and wishing you had a Phd in TeX editing.

TikZ has exactly the same problem. I understood the tutorials and the manual well enough, but I wanted to create class diagrams with it. I got lost quickly and the 500 page manual didn't offer a straight-forward way either.

In the end, I installed the latest version of Inkscape and created all my illustrations with it. Well, actually I always tried to create them with TikZ first, got frustrated after losing an hour or two, and spent another hour creating them in Inkscape.

While Inkscape makes it harder to change an illustration later on and there is no way to layout diagrams automatically, you can still become quite fast with it after learning a few hotkeys.

I wanted to use Eclipse and Omondo's UML plugin to create some diagrams of a Java subproject and it failed horribly..

Having to wait 20 minutes, while it hogs your computer at 100% CPU usage and 400 MB memory usage, to create a UML model of a 5 classes project is quite unacceptable!

Starting Eclipse later crashed it every time, too, by the way. I had to uninstall the damned plugin for Eclipse to start at all again.

In the end I recreated the classes in C++ and used Visual Studio's class diagram viewer to display the classes. You can then print the diagram to PDF and edit it some more in Inkscape, which supports importing single-page PDFs.

I hope I'll find time to write about game development related topics again in the next weeks. There are a few things I want to work on, but sadly university starts on Monday again and it will probably use up most of my time.

Cheers,
Andreas

Sploidz Revisited (Unofficially)

Andreas 'BlackHC' Kirsch — Sun, 23 Aug 2009 09:08:26 +0000

I've already written about the semi-conductor project and how I've written some Flash animations/applications for it. Of course, I'm more interested in making fun stuff¹, so I've decided to put my knowledge to good use and write a small game to see how difficult/awkward Flash actually is.

To sum it up, it is somewhat awkward, at least if you use the IDE itself. FlashDevelop still is as nice as ever, but you can quickly develop games nonetheless. I prefer Torque Game Builder though in retrospect.

Before I continue talking about the development itself, let's take a look at the actual game. Sploidz was the first game I wrote using Torque Game Builder for Joshua Dallman, and since I still had all the assets in my subversion repository, it was an easy decision to try and port this game. If you want to play it, you can download it for free here. I haven't ported everything: I've just rewritten the main characteristic features that make up Sploidz's code in ActionScript.

Without further ado, here is the game:

Click to open Sploidz in its own window

Because the art is still copyrighted and I haven't heard back from Joshua yet, I decided to create a free version that only uses "coder art" :-)

Mr. jRAD has said that this version looks cuter, decide for yourself:

Click to open SploidzCC in its own window

Below you'll find a description of the development and at least one helpful trick and most importantly a link to the source code of the "copyright-free" version.

Because the orginal version is way too difficult to be really fun, I actually sat down one more time and added code to make the platform slower if you're in danger of losing (up to 3 times slower):

Click to open SploidzMoreFun in its own window

Development

I think I spent less than 20 hours on the project. It's hard to say because I worked on it on three different days, but only probably less than 6 hours on each.

The main problems during the development were:

How do I get the art assets into Flash?
How do I use the IDE and objects in a way that is helpful?

Lessons learned:

Even though the project isn't that big, it didn't take long for annoying bugs to appear - mainly in the matching code, because it's somewhat asynchronous and you have to make sure that an item doesn't get removed between events, etc. So again, I can only say that assertions and lots of debug messages are your friend and are really useful for understanding the sometimes mysterious ways of non-obvious code interaction.

Again one of the biggest headache was how to make the game a game and this resulted in implementing the platform game element from the original Sploidz game, although I wanted to avoid that in the beginning because of the added complexity. Roy pointed out though, that if you have 8 columns and only 5 different item types, the main idea of matching items becomes trivial. It's obvious, but I didn't think of it while coding the proof of concept code. Of course you have to somewhat tune the speed and animation durations of everything to make it playable. The game is really hard at the moment, but that's better than too easy and I especially added the "survival timer" in the upper right panel to allow for some competition and personal best times. I might change the platform code at some point to be more player-friendly, but I have more pressing projects to do at the moment: my Bachelor Thesis... /o.

As said above, I actually sat down and added a time scaler to the platform. The code is quite simple but helps the game a bit (I managed to survive 2 minutes and it was more challenging and less depressing to play the game ^ ^). The code is quite simple:

public function getPlatformTimeScale():Number {
    var maxRow:int = 0;
    for ( var column:int = 0 ; column < Grid.numColumns ; column++ ) {
        var firstEmptyRow:int = grid.getFirstEmptyCellRow( column );
        if ( firstEmptyRow > maxRow ) {
            maxRow = firstEmptyRow;
        }
    }

    const minScale:Number = 1.0;
    const maxScale:Number = 3.0;
    var scale:Number = minScale + (maxScale - minScale) * Math.pow( maxRow / (Grid.numRows - 1), 8 );
    trace( "Using platform scale: " + scale );
    return scale;
}

As you see, it uses a power function to make the platform slower only if really few empty rows are left.

Interesting things to know

If you want to important a sequence of images (say you have an animation stored as picture frames diamond_black00.png to diamond_black15.png, only select the first image when "importing to stage" in Flash, otherwise you won't get the dialog asking you whether you want to import an image sequence as animation.
If you have animations that stores multiple frames in one image, you'll have to split the images up into the respective frames. To do this in Photoshop make use of "guides" and slicing and notice the "Slices from Guides" button that is offered. It's really helpful. You can also store the creation of the guides and the slicing as action/macro and reuse it which also saves some time.
If you ever get the error "type was not defined or was not a compile-time constant", you might have a symbol instance that has the same name and class name. Flash freaks out for some reason and displays this weird error message.

For added fanciness, I've also included two older builds that show the evolution of the game code.

Note: There are bugs in these builds, so they are really just meant to show the progress on the first and second day.

Click to open Sploidz0810 in its own window

Click to open Sploidz0813 in its own window

Fun note: I implemented most of the fancy features (combo matches, the score bubbles, the platform) today in a few hours (a lot less than expected). I think ActionScript is an okay language, although it lacks quite a few comfort features of regular modern programming languages (eg function overloading or delegate types that actually check parameter type).

Last but not least here is the link to the code: FlashySploidz Source I release the code itself under GPL and the art under Common Creative.

Over and out, Andreas

of course, the math stuff is also fun from a different point of view↩

More Book Reviews and re: Design Patterns

Andreas 'BlackHC' Kirsch — Tue, 18 Aug 2009 08:44:12 +0000

Pattern-Oriented Software Architecture - A System of Patterns (PoSA I)

I've finally finished Pattern-Oriented Software Architecture - A System of Patterns and I want to say a few words about it.

First I don't think, it's a must-buy. It's okay to read (but more about that in a minute) and certainly it can come in handy to look up patterns while developing, but if you have small budget, it's better to get it from a library.

The book can be roughly divided into 4 parts: architectural patterns, design patterns, idioms, "trivia".

The chapter about architectural patterns is the most interesting, where as the chapter about design patterns is okay, too, but also borrow or expands upon lots of concepts from the GoF book. The idioms chapter only contains one idiom (think small, language-specific "design pattern", e.g. reference counting in C++).

The "trivia" part contains information what pattern are, who came up with the idea, influences, the community, yada yada. The most interesting topic being the community and links in there.

The book itself is quite boring to read and the patterns are described in endless detail, which is good, if you want to look things up and avoid common mistakes. On the other hand this makes it a book, you should not read front to back, because it doesn't really make sense and you won't get a lot out of it either. (I actually read it front to back, and from that experience I draw my advice :-|) Just skim over the patterns, until you find something you don't know and then read the paragraphs that are interesting until you think you understand the pattern and then read some more to verify that you really understand it, but you can probably skip 40% (or more) of the text, if you just want to learn about new patterns and not go into all the details.

Next I'll try to write a few words about each pattern, so I (and maybe you, too) can look up the pattern here instead of in the book.

Note: I just found an awesome page, http://vico.org/pages/PatronsDisseny.html, that contains information about all patterns in the book, so it doesn't make sense for me to write about each and every one of them.

Architectural Patterns

Layers

To sum up 20 pages: it's good to use a multi-layered architecture in your software.

Pipes and Filters

This is something Pragmatic Programmer mentioned, too: Use and support the usage of your tools through pipes and filters. This is mainly something that is common on Unix, but it's incredibly useful and if you support some standard text I/O helps interoperability with little cost.

Blackboard

Finally an interesting pattern, it's also described in Pragmatic Programmer, but you can read the description here, too, because it's a quite interesting pattern.

Broker
Model-View-Controller
Presentation-Abstraction-Control

This one is interesting and worth having heard of. It's similar to MVC above, but more general.

Microkernel
Reflection

Design Patterns

Whole-Part
Master-Slave Anyone who has ever done parallel programming with MPI has used this pattern extensively
Proxy
Command Processor A more elaborate version of the Command patten in the GoF book
View Handler The pattern describes how to design a document/view manager
Forwarder-Receiver
Client-Dispatcher-Server
Publisher-Subscriber

Idioms

Counted Pointer

Now that I'm done with this, I want to share a few useful resources mentioned in the book:

http://www.hillside.net/patterns - contains some useful links
http://www.c2.com/ppr/ - contains a few pattern descriptions

Personally idioms are very interesting for me, too, because the day-to-day coding work is in a specific language and idioms are exactly about that.

A little googling already turns up lots of useful sources. For example:

There are a few more pattern books that I plan to read, so let's see how that turns out..

A Manual for Writers of Research Papers, Theses, and Dissertations

This is an awesome book, that explains a lot about how to write research papers and also about all the little details you should pay attention, too. It's a good read - I spent 1.5 hours a day for a bit more than a week to read through it - and I think it's going to be quite useful, when I start writing my Bachelor Thesis in a few days.

I'm currently writing some ANTLR grammar and StringTemplate stuff and I might write something about that, too, on the weekend.

I'm not sure I'm really happy with ANTLR, but it's probably the best thing out there at the moment, but I'm totally in love with StringTemplate, even though you might say it has some rough edges, too.

But more on that on another day.

Over and out,
Andreas

PowerPointLaTeX Update

Andreas 'BlackHC' Kirsch — Wed, 05 Aug 2009 00:48:34 +0000

Because people complained to me about the formula feature in my PowerPointLaTeX add-in, which used a somewhat experimental approach to editing formula objects by adding an editing text shape that contained the formula code and that would be merged back into the formula as soon as you deselect it, I decided to rewrite it to use a standard modal dialog to edit formula objects:

Updated Ribbon (above) and Formula Editor Dialog (below)

The editor isn't perfect (yet), but it certainly shouldn't add any bugs to the add-in and solve some natural issues the old approach created.

Implementation Note

The idea was pretty straight-forward but the actual UI design was a PITA due me not knowning the panel/flow/table layout concepts very well and the code still has some annoying quirks with auto-scroll, so I need to fix that later.

I almost rewrote the whole cache system, because I'm using a background thread for updating the preview (if the text is changed, a 500 msec timer is started which triggers an update) and the update accesses the cache system, which in turn accesses PowerPoint to return some data, which in turn is busy because of the modal dialog -> dead-lock.

The solution to this is very simple but was not obvious to me at first (I actually began to rewrite the cache system with a feeling that there should be an easier solution): The background thread needs an Invoke call to update the preview picture because the control has been created by a different thread (the main thread) and the code to get an updated picture can simply be moved into Invoke delegate function.

This solved all my problems and made 4 hours of previous work and thinking about a new cache system obsolete :-|

Download the new build at: http://code.google.com/p/powerpointtools/downloads/list

Cheers,
Andreas

Semi-Conductor Optimization (Uni Project)

Andreas 'BlackHC' Kirsch — Sun, 02 Aug 2009 12:29:55 +0000

I've written my last exam yesterday (except for two oral exams in September), so now I have got some spare time before I start working on my Bachelor Thesis tomorrow and I want to use it to wrap up a few things.

During this term I took part in a course that was both a (research) project/presentation/lecture thing, which was fun but also a lot of work. I've already written about one mathematical aspect of it in my post about Analysis, Cauchy-Schwarz and Reciprocal Sums.

The project was about optimizing semi-conductor wiring placement. We wrote a small paper about our findings and the work it was based one - you can look download it here.

We also created a self-running presentation that doesn't contain any Maths at all but makes heavy use of Flash animations (which were exported to .gif manually, which was a huge pain in the ass, which I will never do again if possible) to visualize all the concepts and algorithms.

You can download a PowerPoint 2007 (.pptx) version here or one that works with PowerPoint 2003 here.

For the Student's I sat down and wrote a small Flash application to show the algorithms at work. It's not obvious how it works, so let me explain the major points:

On the right you have a number of panels that you can enlarge by clicking on the small button in the upper right of each panel.
The upper three panels show different views of the same dataset. They will all be updated as you run the algorithm step by step.
The fourth panel lets you change the number of wires and/or their activity.
The last panel shows the electrical field that is created by one active wire in the center. It was created using MatLAB. I've also uploaded the script here.

Abitag

Last but not least I've also uploaded the current version of all my .fla and .as files. You can download it here.

ActionScript is a nice language and you can quickly learn it using the available resources from Adobe. While ActionScript 2.0 is arguably weird, ActionScript 3.0 is quite logical and it's syntax is straight-forward and consistent, too. You can't say that about the IDE (Flash CS4), which is braindead, but if you're only interested in writing ActionScript code, FlashDevelop is an excellent and free alternative.

This is it for now, maybe I'll play around with Flash some more another time. Cheers,
Andreas

From Vertex Positions to Packed Element Arrays

Andreas 'BlackHC' Kirsch — Sat, 18 Jul 2009 17:42:16 +0000

At my new workplace at university I'm currently porting an advanced terrain rendering engine from DirectX to OpenGL. One of the performance optimizations the engine uses is that it draws the terrain tiles right from the index buffer without using a vertex buffer at all - that is it packs the vertex position into the 32-bit index and unpacks it in the vertex shader.

Why is this faster than rendering using a vertex buffer and no index buffer?

When using an index buffer the graphics card can make use of a cache of already transformed (vertex-shaded) vertices and when an index is reused, it can use the cached result instead of running the vertex shader again. Of course, this only works if there exists a certain temporal locality, but that is given. If no index buffer is used, the vertex cache won't be used, because the implicit index is different for each vertex.

Since I need to port the engine from DirectX to OpenGL, I did some research to see if it is possible to do the same in OpenGL. It's not really possible but you can achieve something quite similar in OpenGL 3.0.

Using the Input-Assembler Stage without Buffers (OpenGL)

This is meant as OpenGL analogon for Using the Input-Assembler Stage without Buffers (Direct3D 10).

I think the title is self-explanatory but for greater clarity let me rephrase it: The aim is to render something without using vertex or index buffers, that is (in OpenGL speak) using neither vertex data nor an elements array to render something. Instead the automatically supplied gl_VertexID attribute (vertexId in DirectX) is used to determine the vertex the shader is currently processing.

The example in MSDN simply draws a triangle using vertexId:

VSOut VSmain(VSIn input)
{
    VSOut output;

    if (input.vertexId == 0)
        output.pos = float4(0.0, 0.5, 0.5, 1.0);
    else if (input.vertexId == 2)
        output.pos = float4(0.5, -0.5, 0.5, 1.0);
    else if (input.vertexId == 1)
        output.pos = float4(-0.5, -0.5, 0.5, 1.0);

    output.color = clamp(output.pos, 0, 1);

    return output;
}

If you want to do the same thing in OpenGL, you have at least two problems:

OpenGL uses glVertex\* as end marker for the specification of a single vertex and the array rendering commands (glDrawArrays, glDrawElements, etc.) all require an enabled vertex array.
gl_VertexID is only supplied if:
- the vertex comes from a vertex array command that specifies a complete primitive (e.g. DrawArrays, DrawElements)
- all enabled vertex arrays have non-zero buffer object bindings, and
- the vertex does not come from a display list, even if the display list was compiled using DrawArrays / DrawElements with data sourced from buffer objects.

(from GL_EXT_gpu_shader_4)

There is no way around these requirements, but what you can do is to create dummy vertex buffer with one element, bind it as vertex array and simply draw as many vertices as you want. If you don't access gl_Vertex there is no way that uninitialized data can affect the shader and although behavior is generally undefined in OpenGL, if you render beyond the vertex buffer size, it has worked so far that I've test this on.

Drawing a circle without accessing gl_Vertex

You can download the source code here.

Packing Vertex Positions into the Elements Array

The next step is to start packing and unpacking data in the gl_VertexID. For this, an integer type and bit operations (shifting and masking at least) are required in the vertex shader, so it requires GLSL 1.30 at least.

The code is quite short from my proof of concept project, so I'm pasting the shader here:

#version 130
#extension GL_EXT_gpu_shader4 : enable

out vec4 color;

vec3 unpackVertex(int index) {
    return vec3( index & 0xFF, (index >> 8 ) & 0xFF, (index >> 16) & 0xFF ) * (2 / 255.0) - vec3(1.0);
}

void main()
{
    vec3 unpackedData = unpackVertex( gl_VertexID );
    gl_Position = vec4( unpackedData, 1.0 );
    color = vec4( (unpackedData + 1.0) / 2.0, 1.0 );
}

In main.cpp the equivalent can be found for setting up the elements array:

#define packFloat(v) (int(((v) + 1.0) / 2 * 255) & 255)
#define packVertex(x,y,z) (packFloat( x ) + (packFloat( y ) << 8 ) + (packFloat( z ) << 16))

void display(void) {
    [...]

    unsigned indices[] = {
    packVertex( 0.0, 0.0, -1.0 ), packVertex( 1.0, 0.0, -1.0 ), packVertex( 1.0, 1.0, -1.0 ),
    packVertex( 0.0, 0.0, -1.0 ), packVertex( -1.0, 0.0, -1.0 ), packVertex( -1.0, -1.0, -1.0 )
    };
    glDrawElements( GL_TRIANGLES, sizeof( indices ) / sizeof( *indices ), GL_UNSIGNED_INT, indices );

    [...]
}

For the code to actually make sense I should initialize an index buffer/elements array buffer in OpenGL and upload the indices into it but this was just for testing.

Drawing two triangles by packing their position into the index buffer

You can download the source code here.

Note:

It has been brought to my attention (thanks Yagero), that on NVIDIA cards the driver currently reduces the element array data size from int to short, which causes the third byte (ie z values in this packing scheme) to always be 0.

A fix is to use an element array buffer (index buffer) which prevents the driver from messing with the data size.

You can download the adapted source code here. To show that it works, I have swapped the x and z component in the packing scheme.

Fast Forward

Andreas 'BlackHC' Kirsch — Sat, 18 Jul 2009 14:50:42 +0000

I've quite a few things I've wanted to write about a long time ago and I actually started working on them and taking notes, etc. but never found time for one reason or another to write and publish the actual posts. So this is meant as fast forward of all these text bits.

Movies and Games

Memento

One awesome movie. I've seen it twice already and I still love the movie and its plot. I don't want to spoil too much of the story, but some will probably inevitable.

A few bullet points:

Telling a story in reverse is pretty cool in itself and perfect for the movie
I wonder if one could use this kind of plot for a game, too, but it would make the plot quite linear because the player could only do things that won't change the outcome of the future (i.e. what he has already played).
If the game is linear though and goal based it could be possible easily because you would replay it like a movie. If it's plot-centric, it could be fun and create a kind of suspense similar to Memento.
Fahrenheit would be a game that is plot-centric like that..
The movie depicts a chaotic system: small imprecision leads to huge consequences ("Don't believe his lies").
One of the impressions I liked most is the way you have to constantly reevaluate everything you have seen so far because of some new piece of information from the past which totally changes the whole movie a whole time.
It's kind of difficult to remember the movie at first because the human brain is not used to this presentation of causality (seeing the effect before the cause).
The movie gets you thinking about what defines yourself - what are you if you can't remember things anymore and how do you define yourself through your memory.
A few weeks ago I read a psychology case book that contained a case similar to Memento. Alcoholics sometimes damage their brain through alcohol abuse and lose all ability to memorize anything new. Their long-term memory only works up until some point of time and after that they won't remember anything. They constantly live at that moment and will be lost forever.

Fahrenheit

Fahrenheit is a pretty cool. Like many movies it is excellent for 90% of the playtime and then it suddenly starts to suck and/or becomes very weird story-wise.

It's not a typical game as more a cinematic experience that does a good job at combining gaming aspects with a very advanced plot and some pretty awesome action scenes.

I've really enjoyed the game and just like Omikron: The Nomad Soul (an earlier game by developer Quantic Dream) it's positively refreshing and different.

Some random notes:

Vista compatibility sucks. I had to download a hacked binary to make it start at all on Vista. Otherwise it ran fine except for one crash that was due to my notebook overheating slightly.
Like in many other games you can't skip cut scenes or dialogs easily which is annoying if you just want to replay a chapter up to a certain point
More annoyingly it seems that if you replay an earlier chapter, you have to replay the ones following it, too. Be careful with that :-|
As said - the story is awesome until you have played 90% of the game then it turns a bit into being on the bizarre side of things[[:especially the internet "A.I." :o]].
Also "Hiding at Tiffany's" is awful. If you ever play the game be ready to replay it a few times: You have to hide at Tiffany's place from someone who is searching for, and you have 30 seconds or so to find a hiding place, after which the person starts searching for. The problem is that you don't know where the person will search for you and if you are caught, you obviously won't see where he would search afterwards, so you have to replay the same part quite a few times if you have bad luck... it was one of the more frustrating parts of the game.
Zero Punctuation has a good review of the story issue of Fahrenheit hidden in his review of Condemned 2, it is also funny. It's certainly worth watching :)
The way the controls work in Fahrenheit is also pretty interesting. The wikipedia article about Fahrenheit has a good description of it in its Gameplay section.

Books

The Pragmatic Programmer

Last year I think I wrote that I had started reading "The Pragmatic Programmer". I actually finished reading it quite a while ago, but here are a few remarks about it:

It's a good and nice read and the book contains lots of helpful suggestions and things to keep in mind when coding or designing software or even just when you want to communicate with co-workers, etc.
It's a "common sense" book - similar to Code Complete - and when you read it, you'll often think "that's straight-forward" or "that's the logical thing to do", but it still valuable to have all that common sense written down somewhere and to be able to look at it now and then in search for inspiration.
It's not as useful as Code Complete though, which was a real eye opener (and still is) and it's not going to improve your coding style a lot or the way to think about code design.

OpenGL Superbible

I've bought the "OpenGL Superbible" and it's a pretty good book if you want to learn OpenGL or read a light text about certain advanced OpenGL features before rolling up your sleeves and digging around in the extension specs. It's written like a big and pretty complete tutorial and the latest edition is a lot better suited for the new features than, say, the latest edition of the OpenGL Programming Guide (which is pretty horrible - I've read through the sixth edition and it's pretty much the second edition plus a paragraph tacked on here and there and long explanations of deprecated features).

The only part of the book that is really, really weak and totally useless is the part about GLSL and shader programming. It contains a short description about GLSL and while the chapter summary mentions functions like glUniform and co, the function is not mentioned anywhere in the chapter nor does it provide even one example on how to set or access vertex attributes or uniforms, which is essential.

If you want to learn about GLSL and shader programming in OpenGL I can only recommend the OpenGL Shading Language and the GLSL language specifications.

Multi-Core Programming

First don't buy this book. It's from Intel Press (you can read the book description here) and it's ridiculously expensive for the content it provides.

I got it for free at university presentation from Intel and have read through most of it in the last weeks and really - if you want to learn about OpenMP and threading techniques and tools, there are better sources available online for free.

Real-Time Rendering

This book on the other hand is awesome - buy it if you are interested in computer graphics and want to understand the underlying principles better. It's well-written and presents lots of advanced computer graphics topics in a very understandable way. Especially the chapters about local and global illumination and the physical base of them are very good. It is good starting point to look for resources and papers and the book's homepage is also pretty useful: http://realtimerendering.com/.

Extending Java and Javac

Andreas 'BlackHC' Kirsch — Sun, 21 Jun 2009 00:07:19 +0000

Today I want to write about something I've been working ages ago - specifically in March I wanted to see if I can extend a Java compiler to support LINQ expressions, too.

I probably spend more time on finding a good open-source compiler to experiment with than I later spent on trying things out, so let me share my preferred source with you: http://openjdk.java.net/ is a good address to start with. More specifically http://openjdk.java.net/groups/compiler/ contains some valuable information about the way the compiler works. A nice thing is that there is a branch that has added support for ANTLR which makes adding new language features a tad bit easier as you get to change a grammar file instead of having to tweak hand-written lexers and parsers. More info about it can be found at http://openjdk.java.net/projects/compiler-grammar/. You can download the source code from http://hg.openjdk.java.net/ - don't follow the link to http://hg.openjdk.java.net/compiler-grammar/compiler-grammar, that one will only allow you to download part of the branch (and nothing interesting either, which was very frustrating at the beginning).

I didn't come around to add support for LINQ in the end, but to get known to the compiler and the ANTLR grammer, I added support for the var keyword as known from C#, which allows for automatic type deduction and for anonymous objects (again using the C# syntax). Thus my changes allowed for the following to compile and execute correctly:

public class Test {
    public static void main(String[] args) {
        // automatic type deduction
        var t = Math.atan(1);
        System.out.println( t );

        // anonymous type
        var i = new { Amount = 108, message = "hello" };
        System.out.println( i.Amount );
    }
}

Automatic Type Deduction

Let's take a look at how I added support for the var keyword, which requires an initializer at the variable declaration and automatically deduces the type used and uses that.

This itself is not a feature that I'd actually recommend using in general because it obfuscates a variable's type[[:obviously...]] and makes it harder to read and understand the code but in conjunction with LINQ and anonymous types, it is very useful, because you don't want to know the type of the query[[:and you actually can't if the result is based on an anonymous type]].

The nice thing about ANTLR is that it's.. nice. I'm going post the diff from my repository to show how easy it was to add support for the var keyword in the grammar file:

It originally looked like this:

localVariableDeclaration returns [com.sun.tools.javac.util.List list]
    @init {
        [...]
        }
    @after {
        [...]
        }
    : variableModifiers type
        {
            mods = $variableModifiers.tree;
            type = $type.tree;
        }
    va1=variableDeclarator
        {
            JCExpression ntype = pu.makeTypeArray(type,$va1.i, $va1.dimPosition, $va1.endPosition);
            JCStatement ntype1 = T.at($va1.pos).VarDef(mods, $va1.name, ntype, $va1.tree);
            //pu.storeEnd(ntype1, $va1.stop);
            ptree = ntype1;
            listBuffer.append(ntype1);
        }
    (cm=',' va2=variableDeclarator
        {
            JCExpression ntype = pu.makeTypeArray(type,$va2.i, $va2.dimPosition, $va2.endPosition);
            JCStatement ntype1 = T.at($va2.pos).VarDef(mods, $va2.name, ntype, $va2.tree);
            pu.storeEnd(ptree, $cm);
            ptree = ntype1;
            listBuffer.append(ntype1);
        }
    )*
;

I changed it to:

localVariableDeclaration returns [com.sun.tools.javac.util.List list]
    @init {
        [...]
    }
    @after {
        [...]
    }
    : variableModifiers
        {
            mods = $variableModifiers.tree;
        }
    (VAR |
    type
        {
            type = $type.tree;
        }
    )

    va1=variableDeclarator
        {
            JCExpression ntype = null;
            if( type != null )
                ntype = pu.makeTypeArray(type,$va1.i, $va1.dimPosition, $va1.endPosition);
            JCStatement ntype1 = T.at($va1.pos).VarDef(mods, $va1.name, ntype, $va1.tree);
            //pu.storeEnd(ntype1, $va1.stop);
            ptree = ntype1;
            listBuffer.append(ntype1);
        }
    (cm=',' va2=variableDeclarator
        {
            JCExpression ntype = null;
            if( type != null )
                ntype = pu.makeTypeArray(type,$va1.i, $va1.dimPosition, $va1.endPosition);
            JCStatement ntype1 = T.at($va2.pos).VarDef(mods, $va2.name, ntype, $va2.tree);
            pu.storeEnd(ptree, $cm);
            ptree = ntype1;
            listBuffer.append(ntype1);
        }
    )*
;

The code changes the way a local variable declaration works by using (VAR | type) instead of type and later in the grammar the VAR token is added VAR : 'var'; and a lookahead rule also needs to be adapted[[:trial and error ftw...]]:

localVariableHeader
    : variableModifiers (type|VAR) IDENTIFIER ('['']')* ('='|','|';')
;

The code doesn't enforce that a var variable needs to have an initializer, this is later done in the actual implementation. It would be easy to add a flag to the variableDeclarator but it would require even more changes to the grammar file.

Now we only need to run ANTLR and regenerate the parser and lexer from the grammar and we're done with this part.

The main change is in visitVarDef in MemberEnter (which completes the Enter stage - see http://openjdk.java.net/groups/compiler/doc/compilation-overview/index.html for more info):

public void visitVarDef(JCVariableDecl tree) {
    Env localEnv = env;
    if ((tree.mods.flags & STATIC) != 0 ||
            (env.info.scope.owner.flags() & INTERFACE) != 0) {
        localEnv = env.dup(tree, env.info.dup());
        localEnv.info.staticLevel++;
    }
    // old: attr.attribType(tree.vartype, localEnv);
    // BlackHC: deduce the type from the initializer if we have a variant
    if( tree.vartype == null ) {
        if( tree.init != null ) {
            tree.vartype = make.Type(attr.attribExpr(tree.init, localEnv));
            tree.vartype.type = tree.init.type;
        }
        else {
            log.error(tree.pos, "initializer.required.for.implicit.type");
            return;
        }
    }
    else {
        attr.attribType(tree.vartype, localEnv);
    }

    Scope enclScope = enter.enterScope(env);
    VarSymbol v =
            new VarSymbol(0, tree.name, tree.vartype.type, enclScope.owner);
    v.flags_field = chk.checkFlags(tree.pos(), tree.mods.flags, v, tree);
    tree.sym = v;
    if (tree.init != null) {
        v.flags_field |= HASINIT;
        if ((v.flags_field & FINAL) != 0 && tree.init.getTag() != JCTree.NEWCLASS) {
            Env initEnv = getInitEnv(tree, env);
            initEnv.info.enclVar = v;
            v.setLazyConstValue(initEnv(tree, initEnv), log, attr, tree.init);
        }
    }
    if (chk.checkUnique(tree.pos(), v, enclScope)) {
            chk.checkTransparentVar(tree.pos(), v, enclScope);
        enclScope.enter(v);
    }
    annotateLater(tree.mods.annotations, localEnv, v);
    v.pos = tree.pos;
}

The code simply initializes the initializer expression's type early and sets the variable's type to it.

Because of this Attr's visitVarDef needs to be adapted to avoid recreating the type later, we have to add a hack but since it's prototype code it's not that big an issue. In line 735):

// BlackHC: this if condition is a hack to keep anonymous objects, etc. from breaking >_<
if( tree.init.type != tree.vartype.type )
    attribExpr(tree.init, initEnv, v.type);

Now only an additional line needs to be added to res/compiler.properties to add the error message text that should appear if the initializer is missing and we're done (in line 475):

compiler.err.initializer.required.for.implicit.type=
initializer required for implicitly-typed variables

I also added a line to com.sun.tools.javac.main.Main's compile function to display a custom string to make sure that the correct compiler is run, but that's just cosmetic.

With this, a new keyword has been added to the Java compiler with a few lines being changed only. The compiler itself is not that straight-forward to understand if you're not used to its design, but it's still amazing that it's that easy. It took me about 15 hours at most to implement this feature. 80% of the time was spent looking through the code and grammar and identifying how to best add the keyword and implement it.

If you want to test your compiler something like the following command-line is needed (on Windows)[[:or you can configure Eclipse accordingly..]]:

java -cp MyJavaC\bin;antlrworks-1.2.3.jar com.sun.tools.javac.Main Test\Test.java

Anonymous Objects

This was an even simpler feature to implement that did not require any code changes at all. The change only allows for local var variables but this is just because we only changed the localVariableDeclaration rule. Adding anonymous objects (ie. new { fieldName = initializer [, ...] }) is straight-forward once you have automatic type deduction and if you think about it, it's obvious that it's nothing but a rewrite of new Object() { public var fieldName = initializer; [...] }. ANTLR shows its strength here:

creator returns [JCExpression tree]
        @init {
            [...]
        }
    : 'new' nonWildcardTypeArguments cr1=classOrInterfaceType cl1=classCreatorRest
        {
            [...]
        }
    | 'new' cr2=classOrInterfaceType cl2=classCreatorRest
        {
            createdName = $cr2.tree;
            args = $cl2.list;
            body = $cl2.tree;
            $tree = T.at(pos).NewClass(null, typeArgs, createdName, args, body);
            pu.storeEnd($tree, $cl2.stop);
        }
    // BlacHC: add C# anonymous types
    | 'new' '{' typebody=anonymousTypeBody b2='}'
        {
            createdName = T.at(pos).Ident(names.fromString("Object"));
            $tree = T.at(pos).NewClass(null, typeArgs, createdName, args, $typebody.tree);
            pu.storeEnd($tree, $b2);
        }
    | arrayCreator
        {
            $tree = $arrayCreator.tree;
        }
;

anonymousTypeBody returns [JCClassDecl tree]
        @init {
            ListBuffer<JCTree> defs = new ListBuffer<JCTree>();
            JCTree ptree = null;
            String dc = ((AntlrJavacToken) $start).docComment;
        }
        @after {
            JCModifiers mods = T.at(Position.NOPOS).Modifiers(0);
            $tree = T.at(((AntlrJavacToken) $start).getStartIndex()).AnonymousClassDef(mods, defs.toList());
            if (ptree != null) {
                pu.storeEnd(ptree, $stop);
            }
        }
    : (va1=variableDeclarator
        {
            JCVariableDecl tree = T.at($va1.pos).VarDef(T.at(Position.NOPOS).Modifiers(Flags.PUBLIC | Flags.FINAL), $va1.name, null, $va1.tree);
            pu.attach(tree, dc);
            ptree = tree;
            defs.append(tree);
        }
    (cm=',' va2=variableDeclarator
        {
            JCVariableDecl tree = T.at(va2.pos).VarDef(T.at(Position.NOPOS).Modifiers(Flags.PUBLIC | Flags.FINAL), $va2.name, null, $va2.tree);
            pu.storeEnd(ptree, $cm);
            ptree = tree;
            pu.attach(tree, dc);
            defs.append(tree);
        }
    )*)?
;

This is all that is needed. The code is mostly copy'n'pasted from other rules (classOrInterfaceType and classCreatorRest) and it wasn't really that difficult. The long compile times of ANTLR were the only obstacles when writing it.

Try it

I've uploaded my current sources (ready to compile and run) and you can download the zip here. Just execute the compileAndRun.bat and my javac and the test should be compiled and run.

What Else?

This is it for today. But let me tell you about a few final thoughts:

Hacking away on compiler code and grammar files is a lot fun
It's not feasible for real projects, because you don't want to start questioning the validity of the compiler you're using - I had that with QuakeC and it wasn't fun at all - and chasing compiler bugs is terrible in general when you want to spend your time working on project's actual code
I have started working on a preprocessor that would read in Java code with the extended syntax and emit normal 1.6 Java code. If you want to do something like this, you can probably find a grammar of your language on ANTLR's homepage - for Java it is: http://openjdk.java.net/projects/compiler-grammar/antlrworks/Java.g.
This approach is more difficult though because you suddenly lose the nice functionality that gives you an expression's type for free (which is a non-trivial thing to code on your own if you consider imports and local classes, etc.).
It's best thing to do, if you want to extend the language and it's a good idea to give it a thought if you have a project whose code could greatly benefit from some additional language features that can be easily emulated using normal code, too.
ANTLR is nice for quick prototyping even though the hand-written Java parser and lexer are faster in general.

$\epsilon$ Induction

Andreas 'BlackHC' Kirsch — Tue, 26 May 2009 22:25:48 +0000

Induction over natural numbers is a standard tool in Mathematics. But what about doing induction over real numbers (non-negative real numbers)? The structure is different and it's not immediately clear what is meant, so let me clarify it.

Let $P(v)$ be a proposition that we want to show for all $v \in \mathbb{R}^+_0$.

Then we can use the following "induction principle" to prove it:

\[ \exists \epsilon \in \mathbb{R}^+ \] \[ \text{a)} \forall u \in \left [0,\epsilon \right ]: P(u) \] \[ \text{b)} \forall v \ge \epsilon: \left( \forall u \in \left [0, v - \epsilon \right ]: P(u) \right ) \Rightarrow P(v) \]

It's easy to show that this is a correct way to prove it. I'll use induction over natural numbers to prove it :-)

Let $I_k := \left [k \cdot \frac \epsilon 2, (k+1) \cdot \frac \epsilon 2 \right ]$. Then let's show by induction over $k$ that $\text{(*)} \forall u \in I_k: P(u)$:

Base:

From a) it follows that (*) already holds for $I_0$ and $I_1$.

Induction Step ($k \ge 2$):

If (*) holds for $I_0, ..., I_{k-1}$, then it obviously holds for all $u \in \left [ 0, k \cdot \frac \epsilon 2 \right ]$.

Fix $v \in I_k$, then $v \le (k+1) \cdot \frac \epsilon 2 \Leftrightarrow v - \epsilon \le (k-2) \cdot \frac \epsilon 2 < k \cdot \frac \epsilon 2$. With b) it follows that $P(v)$ holds.

That is, $P(v)$ $\forall v \in I_k$.

It should be possible to show that you can generalize this to: \[ \exists \epsilon \in \mathbb{R}^+ \] \[ \text{a)} \forall u \in \left [0,\epsilon \right ]: P(u) \] \[ \text{b)} \left( \forall u \in \left [\psi(v), \phi( v ) \right ]: P(u) \right ) \Rightarrow P(v) \] with $\phi( v ): \mathbb{R}^+ \to \mathbb{R}^+_0$, $\psi( v ): \mathbb{R}^+ \to \mathbb{R}^+_0$ and $\psi(v) \le \phi(v) < v$.

Analysis, Cauchy-Schwarz and Reciprocal Sums

Andreas 'BlackHC' Kirsch — Thu, 07 May 2009 22:29:48 +0000

Konkrete Analysis

First I want to share some solutions for the exercises of a Maths book. The book is called Konkrete Analysis by Folkmar Bornemann and is written in German as are my solutions for some of the exercises. (I think I cover about 70% of all exercises in the book and pretty much every easy one :-))

I don't claim that my solutions are correct and there are probably quite a few (uncorrected) mistakes, but right now I haven't been able to find any other openly available solutions. Although the book claims to be easily readable without attending the lectures of Professor Bornemann, I doubt that it is possible to do so successfully without being able to do the exercises and while doing so being able to get help or look at possible solutions to find new ideas.

I did all the exercises as way to prepare myself for the exam (it was very successful alas probably not in the most time efficient way), so it also contains solutions for old exams (but those are appended at the end and probably not interesting at all).

You can download the PDF konkrete-analysis-solutions here.

Cauchy-Schwarz and a Problem

The book is quite useful though as was the lecture "Analysis for Computer Scientists" which was based on the book (or vice-versa) and it is probably the single one lecture that has taught most since I started attending Computer Science at university.

One of the many topics (we covered a lot which was one of the nice things) was inequalities and the useful things you can do with them. Especially the Cauchy-Schwarz inequality turns out to be quite useful while being pretty basic:

$< a,b > \leq ||a|| \cdot ||b||$ with $< a , b > = || a || \cdot || b ||$ when $\exists \lambda: a = \lambda b$

Interestingly enough this can already yield results that are quite difficult to obtain otherwise. For example we can easily prove some lower or upper bounds by applying it. That is, we can show the existence of a maximum or minimum of some functions.

To do this, we first obtain a lower or upper bound (for the right or the left side respectively) and then we can use the equality case to prove the existence of the a minium or maximum.

For example, let's take a look at the following problem:

We have \[ x \in \mathbb{R}^n, 0 < x, \sum_{i=1}^n {x_i} = 1 \] and want to minimize \[ \sum_{i=1}^n { \frac 1 x } \]. We simply set \[ a_i := \sqrt{x_i}, b_i := \sqrt{ \frac 1 x_i } \].

With Cauchy-Schwarz we then get:

\[ \sum_{i=1}^n { a_i b_i } = \sum_{i=1}^n { \sqrt{ x_i } \frac 1 {\sqrt{ x_i }} } = \sum_{i=1}^n 1 = n \] \[ \leq \sqrt{ \sum_{i=1}^n \sqrt{x_i}^2 } \sqrt{ \sum_{i=1}^n \frac 1 {\sqrt{x_i}^2} } = \sqrt { \sum_{i=1}^n x_i } \sqrt{\sum_{i=1}^n \frac 1 x_i } = 1 \cdot \sqrt{\sum_{i=1}^n \frac 1 x_i } \]

That is: \[ \sum_{i=1}^n \frac 1 x_i \geq n^2 \].

Now we know an lower bound for the reciprocal sum. To determine the minimal x vector, we remember to start with:

\[ a = \lambda b \Leftrightarrow \sqrt x_i = \lambda \frac 1 x_i \Leftrightarrow x_i = \lambda \]

We use that with the constraint: \[ 1 = \sum_{i=1}^n x_i = \sum_{i=1}^n \lambda = n \cdot \lambda \], resulting in: \[ x_i = \lambda = \frac 1 n \].

Generalization

The nice thing is you can go and generalize this finding to more interesting minimization problems as there also exist more advanced versions of the Cauchy-Schwarz inequality (actually it's Hölder's inequality) adding degrees of freedom to play around with (just one DoF in my case).

Thus it is possible to solve the following class of problems using a slightly more advanced Cauchy-Schwarz inequality:

\[ x,q \in \mathbb{R}^n; 0 < x, q\] \[0 < \alpha, \gamma < \infty; \beta > 0 \] \[\large\sum_{i=1}^n {x_i}^\gamma = \beta \] \[\min \sum_{i=1}^n \frac {q_i} {x_i^\alpha} \]

by utilizing

\[ \large \sum_{i=1}^n { a_i b_i } \le \sqrt[p]{ \sum_{i=1}^n {a_i^p}} \sqrt[{p}']{ \sum_{i=1}^n {b_i^{p'}}} \] with \[ \frac 1 p + \frac 1 {p'} = 1 \]

Equality requires the same linear dependence condition as the normal Cauchy-Schwarz inequation.

I've only deduced the solution for $\gamma = 1$ but it shouldn't be difficult to adapt it once you get the idea how it works (it's fairly straight forward).

You can find the whole deduction in the PDF "Minimum of a Generalized Reciprocal Sum".

For completeness' sake here are the results for an optimal solution $x^{*}$:

\[\large x_i^{*} = \beta \cdot \frac { \sqrt[1 + \alpha]{q_i} }{\sum_{k=1}^n \sqrt[1 + \alpha]{q_k}} \]

\[\large \frac { \left ( \sum_{i=1}^n \sqrt[{1 + \alpha}]{q_i} \right )^{1 + \alpha}} {\beta^\alpha}=\sum_{i=1}^n {\frac {q_i} {{x_i^{*}}^\alpha}}\]

Epilog

Coming up with the solution was actually a lot of fun and quite interesting because I didn't know it was possible to deduce it this way nor did I know how to do it before. If you read the PDF, I think it's quite nice how you can play around with the formulas and come up with new things from the Cauchy-Schwarz inequality.

If you are only interested in $x ^ {*}$, you can shortcut everything by immediately skipping to the equality case once $p$ and $p'$ have been determined and solve for that. I mainly wrote it all out because I wanted to make sure that it actually was correct.

If it wasn't for my Analysis lecture last term, I probably wouldn't have found this.

The Reverse Pigeonhole Principle

Andreas 'BlackHC' Kirsch — Tue, 05 May 2009 00:36:37 +0000

Everybody (I hope) knows the pigeonhole principle.

In short it states that:

If $f: N \to M$ and $n:= \left | N \right | \geq \left | M \right | =: m$, then there exists at least one $m_0$ with $\left | f^{-1}\left ( m_0 \right ) \right | \geq \left \lceil \frac {n} {m} \right \rceil$.

For example, if we have n + 1 pigeons in only n pigeonholes, there is (at least) one pigeonhole that has at least two pigeons inside (thus the name).

One thing I haven't seen mentioned anywhere yet (at least not with this name) is the reverse pigeonhole principle:

If $f: N \to M$ and $n := \left | N \right | \geq \left | M \right | =: m$, then there exists at least one $m_0 \in M$ with $\left | f^{-1}\left ( m_0 \right ) \right | \leq \left \lfloor \frac {n} {m} \right \rfloor$.

Again an example: if we have n + 1 pigeons (it's valid up to 2n - 1) and n pigeonholes, then there exists (at least one) pigeonhole with at most one pigeon in it.

The proof for the reverse principle is simple:

Assume that for all $m_0 \in M$: $\left | f^{-1}\left ( m_0 \right ) \right | > \left \lfloor \frac {n} {m} \right \rfloor$, that is each such set contains at least $\left \lfloor \frac {n} {m} \right \rfloor + 1$ elements, then from $\frac n m - 1 < \left \lfloor \frac {n} {m} \right \rfloor$ it follows that:

\[ n = \sum_{m_0 \in M} { \left | f^{-1} \left ( m_0 \right ) \right | \geq m \left ( \left \lfloor \frac {n} {m} \right \rfloor + 1 \right ) > m \cdot \frac n m = n } \].

This clearly is a contradiction, so the original assumption must have been wrong, which proves the reverse pigeonhole principle.

I haven't seen this written down anywhere before and it's useful to keep it in my mind on its own because it shortens some arguments, so I hope this might help someone :-)

Cheers,
Andreas

PowerPoint Tools

Andreas 'BlackHC' Kirsch — Fri, 03 Apr 2009 09:47:10 +0000

I've finally managed to upload a version of my PowerPoint LaTeX add-in for PowerPoint 2007.

I'm just going to post a short Vimeo video here that shows how inline formulas work (the main feature):

I've tried to work on the add-in during my spring vacation but somehow I have instead spent all my time watching four seasons of House M.D. (which was totally worth it though :-)).

There are still some features like support for MikeTeX, code clean-ups, small bugfixes and a preference window that I should work on, but I don't plan on selling it, so I don't really care if it's still somewhat work-in-progress. I'm going to continue working on it when I have to use PowerPoint again.

You can check it out (and another add-in dubbed Language Painter that I wrote to fix some annoyances when writing presentations in languages different from your keyboard layout) at http://code.google.com/p/powerpointtools/.

"Inter-Mortem"

I'm not really done with the project but I think it's time nonetheless to talk about the project itself. Initially I looked into using TexPoint for a presentation I had to prepare, but it wasn't very user-friendly in my opinion and I also didn't really want to install MiKTeX again, so the idea was born to use a web service (from codecogs.com or wordpress.com) to render the equations and embed the formulas as pictures. A few tests later I knew that it was feasible, so I started to work on the project.

The bulk of the code was written in 4 days during my winter vacation (so that was 4 months ago) - I've only added minor things since then.

This warrants the question what took me so long to upload it and/or write about it. The main reason why it was delayed by such a long time is that I wanted to polish it some more and make sure it really has a well-rounded feature set. In retrospect I shouldn't have added support for non-inline equations because people will hardly use them and the way you edit them is kind of crazy (I create a text shape in the current slide to let the user change the equation) and I reverted lots of the later changes because I wasn't sure which direction to take UI-wise.

The user interface is mostly okay in my opinion but the ribbon might need a redesign to streamline it some more and also maybe come up with new things users would want to do. For that I'll probably wait until I have to use it again (probably in a few weeks' time) and then take notes about what to improve.

The design of the code is somewhat messy when you look at it, but that's because it grew out of a experimental project to see if it was possible to embed LaTeX code the way I wanted, and most of the functionality ended up in one big file/class (LaTeXTool.cs) because everything is centered around inline formulas.

What I (don't) like about C#

Andreas 'BlackHC' Kirsch — Thu, 12 Mar 2009 21:49:54 +0000

I've been using C# personally quite a bit lately and want to share with you a list of the things I like or don't about it.

First the things I like about it (compared to C++ and Java sometimes):

The out and ref keywords are awesome You actually need to use them both when declaring out and reference parameters and in the actual function calls. This really avoids confusion and makes the code clearer compared to the way references are transparently handled in C++.
a class's definition and declaration is kept in just one file Avid C++ coders will probably disagree with me here, but I really like this way of coding things. It allows me to code faster and also reduces one source of compiler errors. To be honest I would probably go insane when coding in C++ if it wasn't for Visual Assist's nifty "Move Implementation to Source File" and "Create Declaration/Implementation" refactorings. Likewise one could say that C#'s (or .NET's) notion of assemblies is better and less messy than C/C++'s compilation units.
Properties are simply awesome Properties look like normal member variables but you can write your own getter and setter functions and also give both different access levels, so e.g. you can have a property that is public for reading but protected for writing. Moreover, you can have C# create the code automatically, so you just end up with a way to specify separate access levels.
Read-only variables can be set anywhere in the constructor If you know C++'s initialization lists, you will thank god for this feature, because this feature gives you a lot more freedom and again results in more readable code (at least in my opinion).
The override and new keywords are really handy Again this is a feature to avoid unnecessary bugs because you get warnings as soon as you "override" a method that doesn't exist.

What I don't like:

No typedefs This really bugs me, because the concept of typedefs is a good way to add abstraction to your code and the using keyword in C# only marginally improves the situation, because it only affects the current module.
No default/optional parameters You have to overload methods instead, which is kind of annoying. I hope this gets fixed sooner or later.
No inner classes like in Java Passing on references to constructors is annoying and inner classes in Java really help it quite often.
No synchronized keyword like in Java Again this is a nice-to-have feature because it would clean up some code.

How to make WordPress display Full Text in RSS Feeds

Andreas 'BlackHC' Kirsch — Sat, 31 Jan 2009 10:46:17 +0000

... and why some WordPress Moderators are idiots

Today I got an email asking me to enable full text RSS feeds - so far so good, only when I activated the "Full Text" option nothing changed - neither in FireFox, nor in Google Reader, nor anywhere else. Only in Feed Proxy I saw a tag appear that contained the full text (including html tags), but the description still only contained the excerpt (without html tags).

Before I link you to the thread that made me conclude the statement in the post title, let me quote from the RSS 2.0 specs:

A channel may contain any number of items. An item may represent a "story" -- much like a story in a newspaper or magazine; if so its description is a synopsis of the story, and the link points to the full story. An item may also be complete in itself, if so, the description contains the text (entity-encoded HTML is allowed; see examples), and the link and title may be omitted. All elements of an item are optional, however at least one of title or description must be present.

That said take a look at http://wordpress.org/support/topic/190901 - having the problem described above and reading the replies, it just left me with one question:

If Otto42 is a moderator, where does WordPress get its trolls from?

The RSS Specs are pretty unspecific and blurry when it comes to the issue, one can at most point to http://www.feedvalidator.org/docs/warning/DuplicateDescriptionSemantics.html, but the main issue I have is that the rants of this "moderator" won't help me fix my problem, because Google Reader and FireFox still don't display the full entries.

A fix for it

If you want to fix it, you can do the following (in WP 2.7):

Open up wp-includes/feed-rss2.php and change

<?php if (get_option('rss_use_excerpt')) : ?>
    <description><![CDATA[<?php the_excerpt_rss() ?>]]></description>
<?php else : ?>
    <description><![CDATA[<?php the_excerpt_rss() ?>]]></description>
    <?php if ( strlen( $post->post_content ) > 0 ) : ?>
        <content:encoded><![CDATA[<?php the_content() ?>]]></content:encoded>
    <?php else : ?>
        <content:encoded><![CDATA[<?php the_excerpt_rss() ?>]]></content:encoded>
    <?php endif; ?>
<?php endif; ?>

<?php if (get_option('rss_use_excerpt')) : ?>
<description><![CDATA[<?php the_excerpt_rss() ?>]]></description>
<?php else : ?>
<description><![CDATA[<?php the_content() ?>]]></description>
<?php endif; ?>

Similarly, if you want to fix your comments, too, open wp-includes/feed-rss2-comments.php and change

<?php if ( post_password_required($comment_post) ) : ?>
    <description><?php echo ent2ncr(__('Protected Comments: Please enter your password to view comments.')); ?></description>
    <content:encoded><![CDATA[<?php echo get_the_password_form() ?>]]></content:encoded>
<?php else : // post pass ?>
    <description><?php comment_text_rss() ?></description>
    <content:encoded><![CDATA[<?php comment_text() ?>]]></content:encoded>
<?php endif; // post pass

<?php if ( post_password_required($comment_post) ) : ?>
    <description><![CDATA[<?php echo get_the_password_form() ?>]]></description>
<?php else : // post pass ?>
    <description><![CDATA[<?php comment_text() ?>]]></description>
<?php endif; // post pass

Seminar about Motion Retargeting

Andreas 'BlackHC' Kirsch — Fri, 30 Jan 2009 23:32:20 +0000

Two weeks ago I had to give a presentation about Motion Retargeting, which I want to share with you now. I created it due to me attending a seminar about the latest developments in Computer Graphics at university and my presentation was about the Siggraph '08 paper "Real-time Motion Retargeting to Highly Varied User-Created Morphologies" from Chris Hecker et al. You can check it out on Chris Hecker's homepage - his website also contains a bunch of other really cool articles and presentations from various conferences, so it certainly is worth taking a look at it.

I've also sifted through quite a lot of IK papers and lectures for my presentation to understand the later part about the IK solver in Spore and I've found a few links that are a nice read:

Chris Hecker's talk about IK is a nice introduction (you also get an audiostream which is nice, too).
Chris Welman's Master Thesis is nice for reading about some of the common methods to solve IK problems.
Steve Rotenberg's Computer Animation course is awesome - the slides about IK are nice, but the ones about locomotion are just awesome[[:they really really are!]]!
Particle IK and particle physics in games are related a lot (the constraint solver is pretty much the same) and for a nice read I can recommend Thomas Jakobsen's Advanced Character Physics.
Pixar's Physically Based Modelling course from 2001 is also nice to get going with physics again - it also explains everything you need to understand the idea behind ODE solvers in rigid body simulations and also has a very nice explanation of stiffness in ODEs.
Principles of Animal Locomotion is a very neat book that is about, well, the principles of animal locomotion, which I have read a bit thanks to Google books and it certainly is quite interesting.
Numerical Robot Kinematics Based on Stochastic and Molecular Simulation Methods is interesting, too, just to see what's possible with particle IK

Ive created a huge PowerPoint presentation for my seminar :-) It includes a few videos (thanks again to Chris Hecker for uploading them and replying to my emails incredibly fast [[:he replied to each and every one of them within 20 minutes, which is crazy fast, if you're used to university communication ;-)]]) and two awesome IK solvers that I've implemented with VBA macros[[:VBA is horrible BTW]] to show how CCD and Particle IK solvers work.

You can find a zip with all the videos and high resolution images here(includes both a PPT 2003 file and a 2007 file). I've also uploaded a small version without videos, macros as PPT 2003 file here, if you don't feel like downloading the 23 MB .zip file.

Here's a video of the two IK solvers:

The CCD solver usually works by solving one bone per inner iteration - I only added a max delta angle to create smooth animations (same for the max delta length in the Particle IK solver).

It seems as if the CCD solver could converge faster if it randomly chose the first joint instead of starting with the end effector joint in every outer iteration.

I've exported the code into an additional IK Playground presentation which contains just one slide and the two IK solvers with the setup you see in the video above. You can find the PPT 2003 version of it here and the 2007 one here.

I've zipped up the macros here if you want to use them in your own slides. I've also written a handy VBA form that allows one to edit everything more easily (the macros are hardly documented though, but if anyone really wants to use them and runs into problems - you can always drop me a line or two in a comment :))

The IK Solver Tool Form

BTW I'm not sure you know about it, but Blender contains an awesome video editor - the UI needs some time to get used to, but the online documentation has improved a lot and with it, it works like a charm. Blender also supports some pretty professional filters, so it's going to be my video editing tool of choice from now on.

Cheers, Andreas

Logical Equivalence

Andreas 'BlackHC' Kirsch — Fri, 30 Jan 2009 22:25:40 +0000

It has taken me 3 months to finally solve this problem---well, actually this is the first time in three months that I have taken a look at the problem again.

Anyway---I proudly present:

\[ (b \Rightarrow B_0) \land (\lnot b \Rightarrow B_1) \equiv \] \[ \equiv (\lnot b \lor B_0) \land (b \lor B_1) \equiv \] \[ \equiv (b \land B_0) \lor (\lnot b \land B_1) \lor (B_0 \land B_1) \equiv \] \[ \equiv (b \land B_0) \lor (\lnot b \land B_1) \lor ((b \lor \lnot b) \land B_0 \land B_1)\equiv \] \[ \equiv (b \land B_0) \lor (\lnot b \land B_1) \lor (b \land B_0 \land B_1) \lor (\lnot b \land B_0 \land B_1) \equiv \] \[ \equiv (b \land B_0) \lor (b \land B_0 \land B_1) \lor (\lnot b \land B_1) \lor (\lnot b \land B_0 \land B_1) \equiv \] \[ \equiv (b \land (B_0 \lor (B_0 \land B_1)) \lor (\lnot b \land (B_1 \lor (B_0 \land B_1)) \equiv \] \[ \equiv (b \land B_0) \lor (\lnot b \land B_1) \]

\o/

VGA Project for Technical Computer Science

Andreas 'BlackHC' Kirsch — Mon, 26 Jan 2009 20:12:56 +0000

Today I'm going to write[[:you'll probably find this as a project page soon, too]] about a project that I've worked on for university together with other students for technical computer science.

Some Introductory Words

Some background information first maybe: everybody has to participate in a small project for technical computer science during the course of their studies at my university. Usually it's just a small project done by a team of three students. I was lucky though and had good marks in the final exams TCS in the first term and was offered with maybe 50 other students to participate in bigger projects that actually work with hardware and have bigger team sizes too (about 10 people usually).

The project started in my second term and due to us being quite lazy students and being not pressured from the faculty itself, only started really working on the project in our summer vacation which explains why we only finished it two weeks ago (mid/end of the third term).

The aim of the project was roughly to display stuff on a monitor by programming an AVR microcontroller as well as an FPGA. We further had a working networking interface (using an old 10 MBit Ethernet card) to retrieve the stuff we should display. We had to write everything (from the Clients to access/connect to our hardware to the VGA signal output code) on our own (except for a uip network stack to use on the AVR microcontroller).

The hardware (which we didn't have to put together¹) looks like this:

We actually started working on and off during the second term, but getting everything setup and running (the FPGA people could probably fill books with horrible stories about Xilinx...) took quite some time and we were incredibly happy when we could connect via network to the AVR for first time and make the little LEDs on the FPGA flash in a pattern.

More than half a year later we were quite happy to produce images like these on the screen though:

The Features

As you can read above the whole project didn't really have a specific outline, it was more like throwing hardware at us and telling us to get creative (and get something done, too, obviously :-)).

We settled on some features when we really started to implement things and had a better intuition about what is possible and what we cannot do:

Java Client to communicate with the hardware
AVR server code only to communicate with the client and network card and transmit/pass through data to the FPGA
FPGA displays the data on the screen (as 80x40 ASCII output on a resolution of 640x480 at 60 Hz)
Fixed color and font tables (ASCII charset and the usual DOS terminal colors)
Custom font and color tables can be sent, too
AVR has a built-in web interface and can be accessed like a website and offers a small Web Client to send and display text messages on the screen (using the default colors and the usual ASCII charset)
Java Client has editors for all three content types (font, color, screen) and can show a preview
Java Client has an Image Converter that can take arbitrary images and create custom font, color and screen data to show them in the best way possible
AVR can also output log/debug messages as html in the web interface[[:that feature only made it in last week when I went back to take some pictures and decided to grab some logs, too :-)]]. Click here and here or here for examples :-)

Moreover there is some unfinished stuff like a hardware emulator to test the Java client with (that was never properly finished) or a batch testsuite to send messages without having to use the Java Client² or the prototype Java Client and the first version of it (different codebase from what I've heard) etc. etc.

You can download the code for the AVR and its subprojects (testsuite, emulator) here.

The Final "Product"

First, let's take a look at what you see after you power everything up:

ETI is the abbreviation for Introduction to Technical Computer Science and GP4 stands for our project (including its number).

Next we have photo of the old terminal we were using to display debug information from the AVR (I spent hours reading it while coding the AVR):

Let's take a look at the web client now:

The Java Client is more awesome though.

I'll just drop a few pictures showing and the results on the real screen:

For the rest I've prepared a litttle slideshow:

The Development

We were a team of ten students, which is quite huge (at least for my taste - and it also turned out to be quite difficult to manage). Three of us volunteered to be the official project leads, ie. the people responsible for keeping an eye on everything, organizing meetings and being the contact persons for Dr. Acher from the TCS chair.

We initially divided us up into 3 teams: an FPGA team, an AVR team and a Java Client team. The Java Client team and the FPGA one were the biggest ones, because we thought most of the work had to be done there (that turned out to be true), while the AVR team only consisted of me (doing all the C programming for the project) and Andreas Hubel who wrote the HTML/JavaScript code for the web client amongst other things. In retrospect the AVR was a lot of work, too, and, of course, underestimated by me, because I thought passing along information from A (the Java Client) to B (the FPGA) could be done in a day, which was not true.

A big chunk of the project was done in 3 days during our summer vacation when we all met and locked ourselves up in the lab and worked on the project from 9 am to 9 pm (at least a few stayed that long..). The rest was coded during various afternoons and sometimes weekends in our third term.

The Team(s)

Java Client:

Martin Klenk
Fabian Immler
Andreas Lang
Lucas Louca

AVR:

Andreas Kirsch
Andreas Hubel

FPGA:

Jan Seeger
Niko Demmel
Andreas Hubel
Fabian Otto

Documentation:

Lucas Louca
Markus Lachinger
Andreas Lang

2 Project Leads

As you might notice in the end we had another dedicated team that created the documentation for the project and we actually only had 2 project leads.

I was one of the project leads and responsible for 20%³ of all emails sent around---I guess I like to talk a lot :>---and had a lot of fun organizing meetings, writing long emails and trying my best not to annoy people too much :-)

What Went Right (speaking as project lead)

Communication was okay and surprisingly good most of the times.
We decided to setup a CMS and Subversion Repository early-on. Jan installed Trac on his server which did both) and the included MediaWiki was really helpful for collecting information and getting everybody up-to-date. We also had a Who-is-Who page with a portrait photo of everybody and their contact details (ICQ, AIM, mobile number, etc etc) which was really helpful quite a few times⁴.
The FPGA team made extensive use of Pair Programming to come to copes with VHDL and a huge state machine that was responsible for the AVR communication back end.
We actually spent some time prototyping things and working everything out before hacking the final code together (though, of course, lots of prototype code wasn't rewritten but just kept and still lives in the current codebase :().
We had a lot of fun and some crazy time together (both good and bad times with weird bugs and long debugging sessions)
All of the planned features worked in the end.

What Went Wrong

Organizing people is very difficult.
Keeping the team together even more so.
The team was a bit too big with some people idling more than others, because the project could have (arguably) been done by (say) 6 people probably.
We changed bits of the communication protocols a few times, which wasn't very nice, because code had to be rewritten (and sometimes even the design had to be changed).
Debugging was a huge PITA in general.
Even more R&D would probably have been beneficial in retrospect.

What I've Done on the Project

Project Lead I think I've spent most time of all with organizing things and writing general project related emails
Wrote all of the AVR C code (3500 LOCs)
Implemented a small HTTP server (a very primitive one)
Implemented a generic state machine for the server which allows to add new message handlers easily
Talked to both the FPGA and Java Client people to figure out how to get data sent back and forth

What I've Learnt

Organizing stuff and trying to make everybody stay on the ball is a lot harder than it sounds
It also needs a lot of time
Even a team of 10 people is huge
If somebody doesn't really do anything, don't wait for a happy ending. Confront people as soon as you notice that and don't let it drag on (- do it in a nice way though). This is something I've learned the hard way in the end, and it's biggest annoyance in retrospect, because some people worked their asses off, while other(s) did not at all..
Low-level hardware programming is always crazy because you're limited in your choice of debugging tools (printfs for me if the stack hasn't been trashed.. >_<) and you experience weird bugs sometimes that no one can even remotely explain (at first).

we actually had to wire the VGA connector ourselves - and the communication channel between the AVR and FPGA↩
some people call me crazy for implementing a command-line parser using CMD's batch language ^ ^↩
62 out of 306 emails and probably some of the longest, too↩
especially for me because I suck at remembering names and - well, it was my suggestion to finally be able to relate names to faces↩

Learn Java Interfaces in 10 Minutes

Andreas 'BlackHC' Kirsch — Sat, 24 Jan 2009 17:44:15 +0000

I've written a small Eclipse project¹ that tries to explain interfaces and their use through commented code examples. It also shows how to make for-each loops support iterating over your classes and how to use the iterator interface to create data to iterate over just in time.

You can find the project here.

Cheers, Andreas

The name is inspired by "Learn Python in 10 Minutes", which is a very nice tutorial that explains everything through code examples↩

WordPress Hacking II

Andreas 'BlackHC' Kirsch — Sat, 24 Jan 2009 12:34:44 +0000

I have a few private pages that I use to store random stuff and ideas and private pages (for a reason I don't understand) don't show up in the pages widget.

It turns out that WordPress's get_pages function always filters them out, while get_posts doesn't (it actually has some logic to figure out whether to show private pages or not).

I've decided to fix this. A real fix would probably be merging get_pages and get_posts because both seem to do pretty much the same except that get_posts is a tad bit more advanced, but I'm all for quick fixes at the moment, because I don't have much time and I don't want to end up considering myself a PHP developer ;-)

To make get_pages return private pages, too, you have to open the file wp-includes/post.php and change the following lines near the bottom of the get_pages function:

$query = "SELECT * FROM $wpdb->posts $join WHERE (post_type = 'page' AND post_status = 'publish') $where ";
$query .= $author_query;
$query .= " ORDER BY " . $sort_column . " " . $sort_order ;

to:

$private_pages_inclusion_where = ";
if ( is_user_logged_in() ) {
    $private_pages_inclusion_where  = current_user_can( "read_private_pages" ) ? " OR post_status = 'private'" : " OR post_author = $user_ID AND post_status = 'private'";
}

$query = "SELECT * FROM $wpdb->posts $join WHERE (post_type = 'page' AND (post_status = 'publish' $private_pages_inclusion_where)) $where ";
$query .= $author_query;
$query .= " ORDER BY " . $sort_column . " " . $sort_order ;

If you also want to make it easier to distinguish private pages from normal ones, you can also change the following bits in wp-includes/classes.php in the method Walker_Page::start_el:

$output .= $indent . '<li class="' . $css_class . '"><a href="' . get_page_link($page->ID) .
    '" title="' . attribute_escape(apply_filters('the_title', $page->post_title)) . '">' .
    $link_before . apply_filters('the_title', $page->post_title) . $link_after . '</a>';

to:

$output .= $indent . '<li class="' . $css_class . '"><a href="' . get_page_link($page->ID) .
    '" title="' . attribute_escape(apply_filters('the_title', $page->post_title)) . '">' .
    $link_before . apply_filters('the_title', $page->post_title) . ($page->post_status == 'private' ? " (Private)" : ") . $link_after . '</a>';

Partial Integration and the Substitution Rule

Andreas 'BlackHC' Kirsch — Fri, 09 Jan 2009 23:21:54 +0000

I only want to write this down to have a place to look it up:

If we want to integrate $\displaystyle{ \int_{a}^{b}{f'\left(x \right) g\left( x\right)} }$ , we can use: \[ \begin{aligned} (f \cdot g)'\left( x \right) & = f'\left(x \right)g\left(x \right) + f\left(x \right)g'\left(x \right) \\ \Leftrightarrow f\left(x \right)g\left(x \right) \big | _ {a=x}^b & = \int _ {a}^{b}{ f'\left(x \right)g\left(x \right) + f\left(x \right)g' \left(x \right) } \, dx \\ \Leftrightarrow \int _ {a}^{b}{ f'\left(x \right)g\left(x \right) } \, dx & = f\left(x \right)g\left(x \right) | _ {a=x}^b - \int _ {a}^{b}{ f\left(x \right)g' \left(x \right) } \, dx \end{aligned} \]

And if we want to substitute a function in an integral:

\[{\int_{g\left(a \right)}^{g\left(b \right)}{ f\left( x \right) } \, dx = F \left ( g \left ( b \right ) \right ) - F \left ( g \left ( a \right ) \right ) = \left (F \circ g \right ) \left ( b \right ) - \left (F \circ g \right ) \left ( a \right ) = } \\=\int_{a}^{b}{ \left (F \circ g \right ) ' \left ( x \right ) } \, dx = \int_{a}^{b}{ f \left ( g \left ( x \right ) \right ) } \, g' \left ( x \right ) \, dx \]

(with $F \left ( x \right ) = \int^{x}{ f \left ( x \right )} \, dx$) or using a trick:

\[\int_{a}^{b}{f \left ( g \left ( x \right ) \right ) } \, dx = F \left ( b \right ) - F \left ( a \right ) = \left ( F \circ g \circ g^{-1} \right ) \left ( b \right ) - \left ( F \circ g \circ g^{-1} \right ) \left ( a \right )=\\=\left ( F \circ g^{-1} \right ) \left( g \left ( b \right ) \right ) - \left ( F \circ g^{-1} \right ) \left( g \left ( a \right ) \right ) = \int_{g \left ( a \right )}^{g \left ( b \right )}{ \left ( F \circ g^{-1} \right ) ' \left ( x \right ) } \, dx = \\=\int_{g \left ( a \right )}^{g \left ( b \right )}{ F' \left ( g^{-1} \left ( x \right ) \right ) \, {g^{-1}} ' \left ( x \right ) } \, dx = \int_{g \left ( a \right )}^{g \left ( b \right )}{ f \left ( g \left ( g^{-1} \left ( x \right ) \right ) \right ) \, {g^{-1}}' \left ( x \right ) } \, dx = \\ = \int_{g \left ( a \right )}^{g \left ( b \right )} { f \left ( x \right ) \, \frac{1}{g' \left ( g^{-1} \left ( x \right ) \right ) } } \, dx \]

(with $F \left ( x \right ) = \int^{x}{ f \left ( g \left ( x \right ) \right )} \, dx$)

Best-of-Explosm Web 2.0

Andreas 'BlackHC' Kirsch — Sun, 28 Dec 2008 14:54:08 +0000

Yahoo Pipes are an interesting concept, as are the other existing mashup tools (like Microsoft's Popfly or Ubiquity), and it is amazing what can be done with a few clicks with them.

Because I've wanted to learn how to work with Yahoo Pipes (before the Google Mashup Editor is released to the public), I've decided to take my best-of-explosm viewer to a different level and prototype it with Yahoo Pipes.

This just shows the first page (it's easier to view - and create - the feed in groups of 20 pictures than all at once). You can follow the link if you want to view more pages.

Stay tuned for more. Cheers, Andreas

Microsoft PowerPoint Rant

Andreas 'BlackHC' Kirsch — Thu, 18 Dec 2008 22:52:04 +0000

I have to do a presentation at university and it needs to be done in PowerPoint - OpenOffice and LaTeX are explicity not allowed. Usually I'm all for Microsoft, but PowerPoint really lacks quite a few things that one would expect to be common in modern presentation software:

ability to insert document properties
working document properties - somehow I get an error message every time I try to open the document property viewer...
changing the language of a presentation (for all slides, etc.) - it's just a huge W-T-F that you have to use macros or add-ins to do that
table of contents slides - it's incredible that there is no way to automate this in PowerPoint.

LaTeX is a 1000x more user-friendly in that regard to be honest. You can download some packages for free that automate everything that is worth automating and you won't be bothered with updating all possible things manually everytime. It also usually supports more powerful slides with advanced navigation, etc.

Nonetheless if PowerPoint sucks, then** Visual Basic for Applications for PowerPoint** is one big brainfuck. The level of stupidity and general suckiness of VBA as language and the fact that you somehow don't find up-to-date documentation about it online (no language specs?) and some braindead decisions in the PowerPoint macro API make it a real PITA to work with..

Anyway, I've written a macro to create a Table of Contents slide automatically for a PowerPoint presentation - it also allows for updating it later on and for customizing its title without overwriting it on update:

' create a Table of Contents slide as second slide (but you can move it around afterwards)
' you can use the macro to update an already generated one without it resetting the title or the slide's position
Const TOCTag = "TOC?Level"
Const TOCSlideName = "TOC?Slide"

Sub CreateTOCSlide()
    Dim contentSlide As Slide
    On Error Resume Next
    Set contentSlide = ActivePresentation.Slides(TOCSlideName)

    If contentSlide Is Nothing Then
        Set contentSlide = ActivePresentation.Slides.AddSlide(2, ActivePresentation.Slides(1).CustomLayout)
        contentSlide.Name = TOCSlideName
        contentSlide.Layout = ppLayoutText
        contentSlide.Shapes.title.TextFrame.TextRange.Text = "Table of Contents"
    End If

    UpdateTOCSlide
End Sub

Private Function FindTOCSlideWithTitle(title As String) As Slide
    Dim cSlide As Slide
    For Each cSlide In ActivePresentation.Slides
        If cSlide.Tags(TOCTag) <> "" Then
            If cSlide.Shapes.title.TextFrame.TextRange.Text = title Then
                Set FindTOCSlideWithTitle = cSlide
                Exit Function
            End If
        End If
    Next cSlide
    Set FindTOCSlideWithTitle = Nothing
End Function

Sub UpdateIndentationFromTOC()
    Dim contentSlide As Slide
    On Error Resume Next
    Set contentSlide = ActivePresentation.Slides(TOCSlideName)

    If contentSlide Is Nothing Then
        Exit Sub
    End If

    Dim contentTextRange As TextRange2
    Set contentTextRange = contentSlide.Shapes.Placeholders(2).TextFrame2.TextRange

    Dim p As TextRange2
    For Each p In contentTextRange.Paragraphs()
        Dim cSlide As Slide
        Set cSlide = FindTOCSlideWithTitle(Left(p.Text, Len(p.Text) - 1))
        If Not cSlide Is Nothing Then
            cSlide.Tags.Add TOCTag, p.ParagraphFormat.indentLevel
        End If
    Next p
End Sub

Private Sub UpdateTOCSlide()
    Dim contentSlide As Slide
    On Error Resume Next
    Set contentSlide = ActivePresentation.Slides(TOCSlideName)

    If contentSlide Is Nothing Then
        Exit Sub
    End If

    Dim contentTextRange As TextRange2
    Set contentTextRange = contentSlide.Shapes.Placeholders(2).TextFrame2.TextRange

    contentSlide.Shapes.Placeholders(2).TextFrame2.DeleteText
    contentTextRange.ParagraphFormat.Bullet.Type = ppBulletNumbered

    Dim index As Integer
    index = 1

    For Each pSlide In ActivePresentation.Slides
        Dim tagValue As Integer
        tagValue = Val(pSlide.Tags(TOCTag))
        If tagValue > 0 Then
            contentTextRange.InsertAfter pSlide.Shapes.title.TextFrame.TextRange.Text & vbCrLf
            contentTextRange.Lines(index, 1).ParagraphFormat.indentLevel = tagValue
            contentTextRange.Lines(index, 1).ParagraphFormat.LeftIndent = 40 * tagValue
            'contentTextRange.Lines(index).ParagraphFormat.Bullet.Type = ppBulletNumbered
            index = index + 1
        End If
    Next pSlide
End Sub

Sub ToggleTOCEntrySlide()
    Dim currentSlide As Slide
    Set currentSlide = ActiveWindow.View.Slide

    Dim newValue As String
    If currentSlide.Tags(TOCTag) <> "" Then
        newValue = ""
    Else
        newValue = "1"
    End If
    currentSlide.Tags.Add TOCTag, newValue

    UpdateTOCSlide
End Sub

Private Function ClampIndentLevel(level As Integer) As Integer
    If level < 1 Then
        ClampIndentLevel = 1
    ElseIf level > 5 Then
        ClampIndentLevel = 5
    Else
        ClampIndentLevel = level
    End If
End Function

Sub IndentTOCEntrySlide()
    Dim currentSlide As Slide
    Set currentSlide = ActiveWindow.View.Slide

    currentSlide.Tags.Add TOCTag, ClampIndentLevel(1 + Val(currentSlide.Tags(TOCTag)))

    UpdateTOCSlide
End Sub

Sub UnindentTOCEntrySlide()
    Dim currentSlide As Slide
    Set currentSlide = ActiveWindow.View.Slide

    currentSlide.Tags.Add TOCTag, ClampIndentLevel(Val(currentSlide.Tags(TOCTag)) - 1)
    UpdateTOCSlide
End Sub









' create a Table of Contents slide as second slide (but you can move it around afterwards)
' you can use the macro to update an already generated one without it resetting the title or the slide's position
Sub CreateContentTableSlide()
    Const contentSlideName = "ContentTable"

    Dim contentSlide As Slide
    On Error Resume Next
    Set contentSlide = Nothing
    Set contentSlide = ActivePresentation.Slides(contentSlideName)

    On Error GoTo 0
    If contentSlide Is Nothing Then
        Set contentSlide = ActivePresentation.Slides.AddSlide(2, ActivePresentation.Slides(1).CustomLayout)
        contentSlide.Name = contentSlideName
        contentSlide.Layout = ppLayoutText
        contentSlide.Shapes.Title.TextFrame.TextRange.Text = "Table of Contents"
    End If

    Dim contentTextRange As TextRange
    Set contentTextRange = contentSlide.Shapes.Placeholders(2).TextFrame.TextRange

    With contentTextRange
        .ParagraphFormat.Bullet.Type = ppBulletNumbered
        .Text = ""
    End With

    For Each pSlide In ActivePresentation.Slides
        If (pSlide.Layout = ppLayoutTitle) Or (pSlide.Layout = ppLayoutTitleOnly) Then
        ElseIf pSlide.Name = contentSlideName Then
        Else
            contentTextRange.InsertAfter pSlide.Shapes.Title.TextFrame.TextRange.Text & vbNewLine
        End If
    Next pSlide
End Sub

The most recent version can be found in this .zip file here (TOC.bas).

Some remarks:

"On Error Resume Next" - it's crazy that VBA uses this to control its error handler (this construct dates back to QuickBasic..)
VBA doesn't support normal exception handling it seems
what does the set statement do and why is it necessary to use it when dealing with object references - I've experienced strange error messages before I started using it
VBA doesn't seem to support a Continue statement (continue as the counterpart to break)
The VBA documentation is inside the Access documentation (in case you look for it)

Hopefully this is useful for other poor souls who have to or try to work with PowerPoint's macro facilities.

Cheers,
Andreas

More WordPress Pimping

Andreas 'BlackHC' Kirsch — Fri, 12 Dec 2008 18:37:31 +0000

Note: this is post didn't make it through the markdown conversion very well. I'm still keeping the explanations, but the examples had to be removed.

PHP and CSS and all that funky web design stuff are funky, because they are so easy to use and you quickly get results. I like it.

I've decided to release my first plugin into the world (it's big hack, but it has turned out to be very useful on my other blog http://info1.blackhc.net).

It supports 3 text replacements:

Xkcd-style Tooltips as Replacement for Footnotes

Example:

[[:This is a default footnote. Hover over it to display it.]]

You can use a different marker, too: Example:

[[:My Footnote#This is a default footnote. Hover over it to display it.]]

Linking to Wikipedia

Example:

[[wikipedia:Pie]]

The link text can be changed, too, of course. Example (you can use : as seperator, too):

[[wikipedia:pi#Fake Pie]]

Expandable Text

Example:

[[expandable#Expandable Text...]]

The link text can be changed, too:

[[expandable#Click me! Click me! Pleeease!#Yay!]]

And we can nest it and it works with other tags. Example:

[[expandable:#Expandable Text...

[[expandable:Click me! Click me! Pleeease!#Yay!]]

[sourcecode lang="java"]<br />
// some Java code<br />
[/sourcecode]

]]

We can also use a Heredoc-like mode: Example:

[[expandable_ex:EOD#Heredoc-like Expandable#[[[[[[[]][[EOD]]

Escaping [[ and ]]

Example:

Escaping [[escape_left]] and [[escape_right]]

WordPress Hacking

Andreas 'BlackHC' Kirsch — Sat, 22 Nov 2008 13:27:20 +0000

Since I've been working on another blog (http://info1.blackhc.net) for my tutorial at university, I've played around with using Microsoft Word for publishing and editing content on the blog. I gotta say that SmartArts are really nice and good way to lighten up blog posts easily.

However Word's support for the current version of WordPress is sub-optimal - that is, it totally messes up formatting when retrieving existing blog posts (that is, Word doesn't recognize any paragraphs or line breaks). I've spent some time digging around in the blog's .php code to find a way to fix it and after an hour or so, I've been able to fix it. It's one-liner I want to share with you:

//'description' => $post['main'], // line 1939 in xmlrpc.php (function mw_getPost)
'description' => wpautop( $post['main'] ),

The function wpautop adds <p> tags to your text (and by default <br> tags, too).

This will fix the formatting bug in Word. This is a hack, of course, but it works for Word.

WordPress has a bug in its Atom Publishing Protocol code, too: It messes up the status code header when asking for user credentials, which prevents your browser from displaying the login form. Again only one line needs to be changed:

// header('Status: ' . $msg); // line 1014 in wp-app.php (function: auth_required)
header('Status: 401 ' . $msg);

Stay tuned for more updates in the next months.. :-) Cheers,
Andreas

PS: University is really keeping me busy..

A Visual Studio Macro Day

Andreas 'BlackHC' Kirsch — Mon, 06 Oct 2008 21:08:32 +0000

Today I decided to continue work on a Visual Studio macro I started developing on Sunday (I mainly worked on it today though - Sunday was just document reading before heading back to Munich and going out with friends).

The samples from Microsoft simply use the comments from MSDN on functions/methods that are part of a class's interface implementation.

I 'documented' a few of my functions by copying the summary out of MSDN but it's a tedious job and consequently I decided to try and automate it. Although Visual Basic 'sucks' or is a least quite some change from writing C/C++/C# code all the time, the Automation model is very powerful and quite nice to use.

My macro provides two methods:

One to add comments to all methods that are part of an interface's implementation
One to add comments to the method the cursor currently resides in

My code currently only adds comments from interface definitions that have been defined outside the current project. It's pretty nifty in my opinion because usually these interface comments won't change a lot and thus it's safe to add them to the source code, while the interfaces one has written themself can still change and the macros can't track that and/or update the comments afterwards.

However, I've added a configuration boolean, so this behavior can be turned off if needed.

My code won't remove or replace comments, it will just add the comment in front of other comments - if the comment doesn't already exist - I've tried to make it quite safe, so code loss or corruption will be avoided.

I hope this code is helpful. I'm releasing it under the Microsoft Public License. If there are good reasons to use a different license, feel free to tell me so :)

Cheers,
Andreas

' Andreas 'BlackHC' Kirsch 2008
' released under Microsoft Public License
Imports System
Imports EnvDTE

Public Module CommentImplementedMethods
    ' we usually only want to use external interfaces because it's safer to assume those won't change anytime soon
    Const UseExternalInterfaceCommentsOnly = True

    Private Function GetSummaryFromDocComment(ByVal docComment As String) As String
        Const summaryStartTag = "<summary>"
        Const summaryEndTag = "</summary>"
        Dim summaryIndex = docComment.IndexOf(summaryStartTag)
        If summaryIndex = -1 Then
            Return ""
        End If
        Dim summaryStart = docComment.Substring(summaryIndex + summaryStartTag.Length)
        Return summaryStart.Substring(0, summaryStart.IndexOf(summaryEndTag))
    End Function

    'Private Function GetInterfaceByName(ByRef classObject As CodeClass, ByRef name As String) As CodeInterface
    '    For Each implementedInterface As CodeInterface In classObject.ImplementedInterfaces
    '        If implementedInterface.Name = name Then
    '            Return implementedInterface
    '        End If
    '    Next
    'End Function

    Private Function GetMethod(ByRef interfaceObject As CodeInterface, ByRef methodObject As CodeFunction) As CodeFunction
        Dim prototypeFlags As Int32 = vsCMPrototype.vsCMPrototypeParamTypes Or vsCMPrototype.vsCMPrototypeType
        Dim prototype As String = methodObject.Prototype(prototypeFlags)

        Dim separationIndex = methodObject.Name.IndexOf("."c)
        If separationIndex <> -1 Then
            Dim interfaceName = methodObject.Name.Substring(0, separationIndex)
            If interfaceName <> interfaceObject.Name Then
                Return Nothing
            End If

            Dim realMethodName = methodObject.Name.Substring(separationIndex + 1)
            prototype = prototype.Replace(methodObject.Name, realMethodName)
        End If

        For Each member As CodeElement In interfaceObject.Members
            If member.Kind <> vsCMElement.vsCMElementFunction Then
                Continue For
            End If

            Dim method As CodeFunction = member
            If method.Prototype(prototypeFlags) = prototype Then
                Return method
            End If
        Next

        Return Nothing
    End Function

    Private Function GetInterfaceMethod(ByRef methodObject As CodeFunction) As CodeFunction
        Dim parentClass As CodeClass = methodObject.Parent
        For Each implementedInterface As CodeInterface In parentClass.ImplementedInterfaces
            If implementedInterface.InfoLocation = vsCMInfoLocation.vsCMInfoLocationProject And UseExternalInterfaceCommentsOnly Then
                Continue For
            End If
            Dim matchedMethod = GetMethod(implementedInterface, methodObject)
            If Not IsNothing(matchedMethod) Then
                Return matchedMethod
            End If
        Next
    End Function

    Private Function GetCurrentMethod() As CodeFunction
        Dim sel As TextSelection = _
            CType(DTE.ActiveDocument.Selection, TextSelection)
        Dim pnt As TextPoint = CType(sel.ActivePoint, TextPoint)

        Dim method As CodeFunction = pnt.CodeElement(vsCMElement.vsCMElementFunction)
        Return method
    End Function

    Private Sub AddCommentToMethod(ByRef methodObject As CodeFunction, ByRef newComment As String)
        If methodObject.Comment.IndexOf(newComment) <> -1 Then
            ' This comment has already been added
            Exit Sub
        End If
        methodObject.Comment = newComment & vbNewLine & methodObject.Comment
    End Sub

    Public Sub CommentImplementedMethod()
        Try
            Dim currentMethod = GetCurrentMethod()
            If IsNothing(currentMethod) Then
                MsgBox("Not inside a method scope at the moment!")
                Exit Sub
            End If

            Dim interfaceMethod = GetInterfaceMethod(currentMethod)
            If IsNothing(interfaceMethod) Then
                If UseExternalInterfaceCommentsOnly Then
                    MsgBox("*External* interface method could not be found!")
                Else
                    MsgBox("Interface method could not be found!")
                End If
                Exit Sub
            End If

            Dim newComment = GetSummaryFromDocComment(interfaceMethod.DocComment)
            AddCommentToMethod(currentMethod, newComment)
        Catch ex As Exception
            MsgBox(ex.Message)
        End Try
    End Sub

    Private Sub TraverseAndCommentImplementedMethods(ByRef elements As CodeElements)
        For Each child As CodeElement In elements
            If child.Kind = vsCMElement.vsCMElementFunction Then
                Dim currentMethod = CType(child, CodeFunction)
                If IsNothing(currentMethod) Then
                    Exit Sub
                End If

                Dim interfaceMethod = GetInterfaceMethod(currentMethod)
                If IsNothing(interfaceMethod) Then
                    Exit Sub
                End If

                Dim newComment = GetSummaryFromDocComment(interfaceMethod.DocComment)
                AddCommentToMethod(currentMethod, newComment)
            End If
            TraverseAndCommentImplementedMethods(child.Children)
        Next
    End Sub

    Public Sub CommentAllImplementedMethodsInFile()
        Try
            TraverseAndCommentImplementedMethods(DTE.ActiveDocument.ProjectItem.FileCodeModel.CodeElements)
        Catch ex As Exception
            MsgBox(ex.Message)
        End Try
    End Sub
End Module

VSTorqueScript #6

Andreas 'BlackHC' Kirsch — Fri, 03 Oct 2008 22:35:32 +0000

I've spent the last two days working on the Debug Engine and I've made some progress:

Tracepoint Support
Breakpoint Support
Support for the Watch window (I might have enlisted this already once but now support is more complete)

I'm not going to post any screenshots this time, instead I'll try to create an alpha package for you to test. This will probably take some time, too, because I want to make sure that you can install and uninstall it cleanly. I've been using the experimental hive so far, so I have to make sure it doesn't corrupt anyone's installation accidentially... :-)

A few implementation notes for interested people as always:

If you ever have to marshal an object to an IntPtr, be very careful, because doing the wrong thing will certainly crash your application - I had that happen to me and it took me quite some time to figure out why Visual Studio was suddenly 'experiencing internal problems' and had to be closed' all the time:

When you have to convert an IntPtr to a wrapped object, use
```
    MyObject object = (MyType) GetObjectFromIUnknown( intPtr ); // or As MyType;
```
As you see, the intPtr is a COM object and every COM object implements IUnknown, so the call is pretty safe. The conversion afterwards is already 'managed', so safe, too. If you want to convert an IntPtr from an object, you have to be more careful though. The Marshal namespace offers at least two methods for it:
```
public static IntPtr GetComInterfaceForObject(
    Object o,
    Type T
)
```
and
```
public static IntPtr GetIUnknownForObject(
    Object o
)
```
Don't use the latter for anything except if you really need an IUnknown interface IntPtr pointer! The method really only returns a pointer to the IUnknown interface of the object, which usually isn't what the unmanaged code expects - which probably will result in misinterpreting some uninitialized memory as a vtable that is not there. You have to use GetComInterfaceForObject and don't forget this. Because you're only working with IntPtrs in the end, there is nothing to warn you about your misstep.
If you want to use an object with System.Collection.Generic.Dictionary<K,V>, it's not sufficient to implement IEquatable<K> as stated in the documentation - you also have to override GetHashCode (and override the normal Equals, too, then while you're at it), because Dictionary uses GetHashCode for the bucket assignments and you can implement IEquatable as hard as you want, GetHashCode might simply assign two "equal" objects to different buffers and you have a problem..
One of the benefits of COM is that no one cares about your implementation.. take advantage of it! I had to remember this the hard way, when I implemented my breakpoint code. At first I wrote a PendingBreakpoint class, which implemented IDebugPendingBreakpoint2, and then a BoundBreakpoint one, which implemented IDebugBoundBreakpoint2. It was unnecessary messy and today I sighed and rewrote the breakpoint code to use just one Breakpoint class that implements both interfaces and it's a lot more elegant now.

Funny fact: An enum that is even mentioned in MSDN and the SDK docs is missing from the Interops library:

public enum enum_BP_UNBOUND_REASON
{
    BPUR_UNKNOWN = 0x0000,
    BPUR_CODE_UNLOADED = 0x0002,
    BPUR_BREAKPOINT_REBIND = 0x0003,
    BPUR_BREAKPOINT_ERROR = 0x0004
};

The initialization process for the DE is as follows:
1. DebugEngine and Program Creation Events
2. Load Complete Event
3. Call CreatePendingBreakpoint for all known breakpoints
4. First Continue Call
5. Entry Point Event
6. Either the second Continue call or the SDM enters break mode
I had to write a few wrapper classes, too, which I might post about next time around. It's enough for today though.

Cheers,
Andreas

Crysis notes - random

Andreas 'BlackHC' Kirsch — Thu, 02 Oct 2008 00:21:15 +0000

Last week I had the joy of playing and beating Crysis (you'll find out later why I emphasize that I beat it). While playing I took some notes, which I want to share with you (some might be funny facts, others might show why Crysis sucks and others could be proof for some awesomeness that you can---still?---find in the game).

First off though, this is going to contain lots and lots of spoilers, so just lemme put up a

SPOILER ALERT

sign here before continuing.

Let me also start with a few clarifications: although it might totally sound as if Crysis is a shitty game when you read my notes, it still is an awesome game with the best graphics, best physics and best destructible game environment you can find anywhere at the moment.

And it looks gorgeous..

Absolutely gorgeous.

Let's start now:

I'm always kind of an explorer in games. After killing everyone in the first map in the part when you have to disable the jammer, I went around and looked at everything. I also began a violent rampage and destroyed everything I could, too. It's fun. To be honest when I played the game for the first time at my cousin's, I spent half an hour destroying huts, cars, boats and trees---lots of them---and I had the most fun in a long time, too. Having destroyed everything I noticed that there was a fridge in the hut, so I opened it and died. You heard me: the bloody fridge killed me when I opened it! Never try to open a fridge in the game when something is behind you and blocking you because the door of the fridge will squish you like a fly. Of course, I hadn't saved, so I enjoyed playing the same part a second time >_<
I also realized that corpses disappear really quickly and there is not ragdoll physics on them either. I had the German version installed (except for the English language paks), so I figured that the German version has been cut, but a quick search quickly showed that Crytek removed some features everywhere and actively banned people who released mods which re-enabled some of them. I'm always pissed when I have to play a cut version because of some weird sense for censorship in Germany (for whatever reason---Crysis is rated AO in Germany, so I honestly don't get it), but the way threads were closed and deleted on all possible Crysis boards really made me angry. Crytek/EA sent emails to boards that contained threads they didn't like and pressured the admins into deleting stuff---someone must have learnt a thing or two from his history lessons, right Crytek? Sometimes soldiers lose their caps and when you shoot these, you actually get to see the original blood decals and splatter and it's pretty crazy how high-detail they are and how much better they look compared to the "blood patches" available.
Playing 10+x intro videos is also incredibly annoying, I removed most of them after starting Crysis for the third time because they were simply annoying.
One thing that is very nice are the suit shortcuts (double-tapping some keys to activate a suit mode, e.g. jump for strength, crouch for stealth, sprint for speed and walk backwards for armor). For the first one or two hours I played without them and when I discovered them in the options menu, the game suddenly became a lot more fun. I still can vividly remember a scene in the first level, where I used the suit shortcuts to do some pretty crazy stunts and runs: Jumping onto a roof, sprinting and jumping towards a jeep and shooting everyone inside during the final jump. Later I kept sprinting towards soldiers like some kind of Predator and used one of them as shield before throwing him onto his colleagues.
Crysis' AI is pretty horribly though. At least the low-level AI stuff: Soldiers didn't see me countless times, although I was standing right in front of them or shooting at them and they simply didn't notice me. Later in the game I found out that the silencers had some weird/exaggerated effect and that turning the silencer weapon modification off actually fixed some of the issues but in general I had lots of problems with it. If you were standing on a roof and actively shoot someone, the bots wouldn't notice it---even when playing on Hard or Delta difficulty.
I also once chased a patrol boat in sprint mode and I was pretty disappointed by the fact that you can't board a boat :-( You can jump onto it by using speed mode, sprinting under water and jumping out of the water like a flying fish---if you time it well, you can actually land on a patrol boat but the AI totally freaks out when that happens: The driver stops the boat, but doesn't do anything else, while the cannoneer jerks around continuously which looks really awkward and buggy.
In general you can have quite a lot of fun by trying weird strategies. For example, there is a command post on a cliff that you have to take out and I drove to it with a boat and jumped up the rocks on the cliff side---only to get stuck between two rocks which eventually forced me to reload the last quicksave, because I couldn't get the player unstuck. The game often punishes you for unconventional actions because the suit makes it really easy to get outside the level designer's thinking which is often equal to a total breakdown of gameplay and fun. I'm used to become creative when I have additional powers like those provided by the nano suit and I felt encouraged to use different approaches to what I'd normally do in a first-person shooter. I think that's just the conditioning from other games where you often have to use special abilities in the levels to solve puzzles (e.g. Prince of Persia or Prey). It's a pity Crysis didn't really require that.
Did I mention that the physics system is awesome? It is. If there is one single element that makes Crysis exceptional than it is its graphics. But only its physics system makes the game a pleasure to watch. The way you can destroy things really adds a lot and the way the environment reacts to your destruction---like sandbags being hurled around by a grenade explosion---make the game more satisfying. I once read that one of the intentions of Crytek was to make the game play and look like a good action movie and I often felt this way when playing. Trying my best to make the action look worthwhile---but often enough the game punished me for that idea.
I actually replayed the second level of the game, because at first I used a boat to avoid most of the level and get to the village where a hostage was kept and it wasn't fun at all because I entered the level from the wrong side (unintended by the game designers) and it was very boring.
I still don't get why anyone can say that Crysis's AI is awesome.. its squad behavior is horrible and soldiers generally die like flies. There was one particular moment when I was inside a house and soldiers kept in pouring one by one from the same entrance (mind you there were windows and they had grenades, too, but didn't use them) and I killed them one by one. This was when I decided to restart the level.

You need to force Crysis to be fun, if you want to have fun.

Because of all the AI issues I decided to try the Advanced AI mod for Crysis and I have to admit that the opponents became a lot tougher but still had issues (again I suspect some silencer weirdness happening). Later in the game I removed the mod again, because I experienced some weird bugs with the viewmodels and decided that I should play the game vanilla, so I know who is responsible for bugs.
In some levels it is also ridiculously easy to get out of the level area and be notified that you will be terminated in 3...2...1...woops if you don't get back---if you toy around with strength mode, it's pretty easy to jump into areas you shouldn't be in. I think the levels weren't tested sufficiently with the different suit modes and/or the guidlines for the level designers weren't good enough.
In the level in which you have to destroy the AAA batteries Crysis became pretty boring for me. Everything was nice but it was repetitive, too, and it wasn't challenging at all although it was still difficult (on Hard/Delta).
The tank level after it is pretty awesome. I got lost a few times though and didn't know what to do because being told to "move forward to alpha front line" or something like that doesn't really help when you have no idea what people are talking about and the map doesn't show anything either (I kind of expected an ETQW-esque map to show up and tell me what to do). The first time I played the level I drove around and felt totally lost because I didn't know what to do and whom to shoot. Then I decided that I wouldn't let either of the accompanying tanks 'die' and restarted the level and stuck to that and although I had to reload more often, it was ok and fun because I had more of a mission than just reaching some train station---instead I took the enemy tanks and positions out one by one till I ran out of shells. GameStar complained about the lack of guidance in the game and I have to surprise myself by agreeing with them on this finding :-/
I noticed some weird viewmodel bugs during the game. Sometimes when looking through the scope and switching back, the view position wouldn't be reset and suddenly you wouldn't see your weapon anymore. Another weird bug was with the actual player viewmodel: The player's arm were suddenly visible behind me holding nothing. Saving and reloading would fix those bugs most of the time.
Another weird thing was my fight against General Kyong: I didn't know what to do at first and I didn't really think of it, when I switched to strength mode and punched General Kyong to death. You heard right: I punched him to death---without losing any health or being hit once. How stupid is that for a boss fight? The poor coward tried to run away from me but he wasn't faster than me, of course, because he was using strength mode, too. He also somehow didn't manage to shoot because his mini-gun has quite some start-up time. That was the most sobering moment in the game for me because what I did felt totally wrong and I should have lost for that. Instead it worked exceptionally well and I punched him some 10 to 20 times before he dropped dead..
The alienship was a meh level, too: it was nice in the beginning and the zero gravity stuff was okay, too, but it was too long to be interesting. At the end, I had to wait like 10 minutes in the last chamber, before something actually happened.
The "Paradise Lost" level looks very nice and I enjoyed the snow and ice, too. I still doubt that anything can actually burn at -200 degrees (F or °C, doesn't really matter either), but whatever.
During the cut-scene in which Prophet reappears the game actually crashed a few times for me (that is it crashed each time I replayed it). I waited some 10 minutes hoping that it was some overheating issue with the graphics card and indeed it worked after that, but good lord, I really was scared that playing Crysis was over for me at that moment.
The carrier levels are okay, although the seemingly endless waves of aliens were tedious after some time. I never get why games that try to appear 'serious' and 'realistic' still employ boss fights and the like as a means of game design. It totally doesn't make sense and only Arcade games should use it. When I played Half-Life 2, the boss helicopter totally put me off, and the same applies to all boss fights in Crysis. It simply doesn't feel right...
Especially if you can't finish the game for some damn bug!!! One thing that really made me angry and with angry I mean annoyed and with annoyed I mean totally pissed off was a bug in the last bit of the last level where you have to shoot a hatch with the TAC gun: The damn thing wouldn't lock on! I replayed the last part of the level for 3 or 4 times on Hard until I was totally upset with the game. Add to that some annoying bug where you can fall through the deck and I gave up. I googled a bit and I found out that quite a few people had actually experienced that bug and it has turned them off as much as me. I switched to god mode, infinite ammo, etc. and replayed the last half of the level to see if I could maybe actually finish the game and indeed it worked when I played it for the fifth time.
The ending totally sucks. Cliff hangers are okay sometimes, but I really felt like it was a really bad ending---especially after that incredibly stupid boss fight.
Last but not least a last bug: If you catch a hunter, it needs quite a few punches to die. If you simply throw it away from you after catching it, it immediately deactivates. Logic?

All in all, the bugs in Crysis and the constant AI issues drove me nuts and I totally felt like I beat the game when it was over and I don't want to touch it anytime soon again. I feel sorry for EA's QA people. They must have been in for a painful one if the released game still had so many bugs :( However Crysis is still a good game (if you have a fast PC) and has some nice cut-scenes and action sequences and the graphics and physics in the game are simple the best you can find at the moment (except for Crysis Warhead I guess).

With this I conclude my 'random notes' review and hope that Crysis: Warhead has fewer bugs and is more fun to play because Crysis really deserved to be more fun.

Cheers,
Andreas

VSTorqueScript #5.1

Andreas 'BlackHC' Kirsch — Mon, 29 Sep 2008 09:04:30 +0000

Sadly there isn't a lot new that I can post since I've hardly had time to work on my pet project during the last week - a few friends from Texas and California came over for the Oktoberfest and it suffices to say that I drank a lot of beer last week.

Nonetheless there is some progress and I've finished support for stepping in break mode (although I still have no idea what enum_STEPKIND.STEP_BACKWARDS is about since I've never heard of stepping backwards before).

There is no screenshot this time because you wouldn't really see a difference, except if I did, before/after screenshots to show that the stepping works.

A few coding notes as always:

The current TorqueScript telnet debugger doesn't really tell you whether it breaks because of stepping or because it has hit a breakpoint, so I have to figure it myself. It is pretty straightforward though because stepping usually implies a certain callstack change (or the lack thereof) when finished:
- "Step Into" implies that the new callstack height > old height
- "Step Over" implies no change
- "Step Out" implies of course: new callstack height < old height

So it's pretty easy to figure out whether the stepping was finished or interrupted by a breakpoint.

The class diagram designer/viewer in Visual Studio 200x is awesome. If you haven't tried it, you've missed a really helpful feature of Visual Studio. Lately I've been using it to comment my public functions and parameters, etc. and for small refactorings (like name changes) in general.

Last but not least I think I've written a neat communication class for synchronous and asynchronous message processing which I'd like to share (if you're going to use it though, please tell me and/or send me some feedback):

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Diagnostics;

namespace BlackHC.VSTorqueScript.DebugEngine
{
    class Communication
    {
        [Flags]
        public enum ReturnAction
        {
            ContinueSearch = 0,
            AcceptMessage = 1,
            RemoveHandler = 2,
            Done = AcceptMessage | RemoveHandler
        }

        /// <summary>
        /// Delegate Communication will call to send a message
        /// </summary>
        public delegate void SendMessageDelegate(string message);
        /// <summary>
        /// Delegate which Communication will call to read a message
        /// </summary>
        public delegate string ReadMessageDelegate();

        public delegate void ReplyMessageHandler(string message);
        public delegate ReturnAction MessageHandler(Communication sender, string message);

        /// <summary>
        /// Events in this list are invoked in reverse order (to allow FILO-style message handler processing).
        /// Lock MessageHandlers when adding message handlers
        /// </summary>
        public event MessageHandler MessageHandlers;

        private readonly ReadMessageDelegate ReadMessage;
        private readonly SendMessageDelegate SendMessage;

        private Thread asyncMessageReceiver;
        private object asyncLock = new object();

        /// <param name="readMessage">Delegate to read messages from</param>
        /// <param name="sendMessage">Delegate to send messages with</param>
        public Communication(ReadMessageDelegate readMessage, SendMessageDelegate sendMessage)
        {
            ReadMessage = readMessage;
            SendMessage = sendMessage;
        }

        /// <summary>
        /// Start processing incoming messages by starting the asynchronous processing thread
        /// </summary>
        public void StartReceiving()
        {
            if (asyncMessageReceiver == null)
            {
                asyncMessageReceiver = new Thread(ReceiveMessages);
                asyncMessageReceiver.Start();
            }
        }

        /// <summary>
        /// End the asynchronous receiver thread
        /// </summary>
        public void Close()
        {
            if (asyncMessageReceiver != null)
            {
                asyncMessageReceiver.Abort();
            }
        }

        /// <summary>
        /// Send a simple message
        /// </summary>
        public void Send(string message)
        {
            SendMessage(message);
        }

        /// <summary>
        /// Send a request and wait for a reply that starts with the expected string. Time-out after a specified period of time
        /// </summary>
        /// <param name="request">Request to send</param>
        /// <param name="replyStartsWith">Start of the reply to wait for</param>
        /// <param name="millisecondsTimeout">Timeout time in msecs (-1 for infinite)</param>
        /// <param name="timedOut">Out variable to be set depending of whether the request timed-out or not</param>
        /// <returns>remaining part of the found message (after the replyStartsWith part)</returns>
        public string SyncRequest(string request, string replyStartsWith, int millisecondsTimeout, out bool timedOut)
        {
            AutoResetEvent waitHandler = new AutoResetEvent(false);
            String reply = "";

            MessageHandler handler = delegate(Communication sender, string message)
            {
                if (message.StartsWith(replyStartsWith))
                {
                    reply = message.Substring(replyStartsWith.Length);
                    waitHandler.Set();
                    return ReturnAction.Done;
                }
                else
                {
                    return ReturnAction.ContinueSearch;
                }
            };

            lock (asyncLock)
            {
                Send(request);
                MessageHandlers += handler;
            }

            timedOut = waitHandler.WaitOne(millisecondsTimeout, false);
            if (!timedOut)
            {
                // remove the handler again, since we timed-out
                lock (asyncLock)
                {
                    MessageHandlers -= handler;
                }
            }
            return reply;
        }

        public string SyncRequest(string request, string replyStartsWith, int millisecondsTimeout)
        {
            bool timedOut;
            return SyncRequest(request, replyStartsWith, millisecondsTimeout, out timedOut);
        }

        // TODO: add a time-out here, too? [9/18/2008 Andreas]
        /// <summary>
        /// Starts an asynchronous request.
        /// </summary>
        /// <param name="request">Message request to be sent</param>
        /// <param name="replyStartsWith">Start of the reply to wait for</param>
        /// <param name="replyMessageHandler">Message handler that is passed the rest of the message (after the replyStartsWith part)</param>
        public void AsyncRequest(string request, string replyStartsWith, ReplyMessageHandler replyMessageHandler)
        {
            MessageHandler handler = delegate(Communication sender, string message)
            {
                if (message.StartsWith(replyStartsWith))
                {
                    string reply = message.Substring(replyStartsWith.Length);
                    replyMessageHandler(reply);
                    return ReturnAction.Done;
                }
                else
                {
                    return ReturnAction.ContinueSearch;
                }
            };

            lock (asyncLock)
            {
                Send(request);
                MessageHandlers += handler;
            }
        }

        private void ReceiveMessages()
        {
            while (true)
            {
                try
                {
                    string message;
                    try
                    {
                        message = ReadMessage();
                    }
                    catch (System.Exception)
                    {
                        // the connection has been closed or something similar has happened, so terminate the thread
                        return;
                    }

                    if (message != null)
                    {
                        ProcessMessage(message);
                    }
                    else
                    {
                        // lost connection, terminate the thread, too
                        return;
                    }

                }
                catch (System.Exception e)
                {
                    // don't let Visual studio die here
                    Trace.WriteLine("Exception in ReceiveMessages: " + e.ToString());
                }
            }
        }

        private void ProcessMessage(string message)
        {
            if (MessageHandlers != null)
            {
                System.Delegate[] invocationList;
                lock (asyncLock)
                {
                    invocationList = (System.Delegate[])MessageHandlers.GetInvocationList().Clone();
                }

                foreach (System.Delegate eventDelegate in invocationList.Reverse())
                {
                    MessageHandler handler = (MessageHandler)eventDelegate;

                    ReturnAction action = handler.Invoke(this, message);
                    if ((action & ReturnAction.RemoveHandler) != 0)
                    {
                        // FIXME: is this lock necessary? [9/18/2008 Andreas]
                        lock (asyncLock)
                        {
                            MessageHandlers -= handler;
                        }
                    }
                    if ((action & ReturnAction.AcceptMessage) != 0)
                    {
                        break;
                    }
                }
            }
            else
            {
                Trace.WriteLine(String.Format("Communication: Unhandled message - '{0}'", message));
            }
        }
    }
}

VSTorqueScript #5

Andreas 'BlackHC' Kirsch — Wed, 17 Sep 2008 10:53:50 +0000

It took me quite long to make progess in the last days. Partially this is due me being in Munich again and working in my uni job part-time again - and I'm stunned each time how tired I am when I come home.

It also is due to me experimenting with my TelnetDebuggerClient implementation that connects my debug engine to Torque's Telnet Debugger. I've tried quite a few different approaches for synchronous and asynchronous communication before I could settle for the simplest and the more advanced ones reside in the repository's history. I'll probably have to switch to them sooner or later because I only implement asynchronous communication at the moment which is fine for most things except when you need more advanced debugging info like for displaying objects properly in the debug window for example.

I've also had to read the "Definitive ANTLR Reference" in one night because I had to return it to uni's library the next day, so I was busy for 7 hours reading 350 pages of a parser reference. It was interesting though and tonight I'll try to write a QuakeC grammar - that's the plan at least.

However yesterday, I had time to work on the debug engine some more and here's a picture that tells it all:

VSTorqueScript now supports displaying the callstack, too

As you see you can now break a program and it will retrieve the callstack from Torque and update the callstack window automatically and open the right file and jump to the correct file position, too - or rather as I've just noticed one-line off because VS line indicies have base 0 and TorqueScript uses base 1 >>..._

A few implementation notes as always:

It seems, if your Debug Engine doesn't implement IDebugEngineLaunch2, your Program Provider has to implement IDebugProgramProvider2.WatchForProviderEvents

Moreover, if you don't implement IDebugEngineLaunch2, which in turn forces you to implement IDebugProcess2 (and maybe IDebugProcess3, too), suddenly some "deprecated" methods in your Program class (your IDebugProgram2 implementation) will be called, which, if you follow MSDN and the debug engine sampe, will result in failed asserts and the nice fail message "This function is not called by the debugger." :-) It's kind of logical that IDebugProgram2.Execute() is going to be called because the debugger has no IDebugProcess2 interface to call _Execute _on..
What is enum_FRAMEINFO_FLAGS_VALUES? It's undocumented and I suspect it's used for the undocumented m_dwFlags field in FRAMEINFO...
I'm totally stunned by how stable Visual Studio is. Most of the interfaces I implement are only half implemented and return E_NOTIMPL on most calls and Visual Studio runs and runs well and only displays the information it can in a clean way.

I guess this is one of the advantages of the COM programming/design paradigma and I think it's great. Being able to implement interfaces bit by bit and add functionality slowly makes debugging easier and also helps you understand things better - which is good because even though a lot is documented, you still need some practice and trial & error to get things right eventually.

Enough is enough though for now, and I'll head to uni in a bit.

Only one last remark: I've started reading the book "The Pragmatic Programmer" and so far it's really entertaining and, I guess, good. I'll write more on it, when I've finished it and can actually assess its qualities..

Cheers, Andreas

Weird Things Happen

Andreas 'BlackHC' Kirsch — Mon, 15 Sep 2008 19:15:17 +0000

Today I was at work for the first time since August and nobody seems to have logged into my shared workstation since then: my account name was still in the login field.

The first weird thing that happened was that suddenly the application I've been working on didn't run correctly anymore.

First let me tell you a bit about the app: It renders a volume texture with some custom raytracing shaders in one window and allows editing of the transfer function - this is a function (or texture) that maps a value in the volume texture to a color in another window. The transfer function is made up of shapes that you can colorize and stretch or deform the way you want to allow for easy editing. Look at this picture to see what a transfer function (editor) could look like - this isn't ours though. The TF editor uses an FBO to render the transfer function into the transfer texture.

While the rest of the app was running mostly fine (that is AntTweakBar rendered fine and the background of the transfer function editor which is a histogram of the volume texture, too), the actual transfer function rendering was corrupted. On my workstation you could move the primitive around and while you did so it would either totally disappear, invert its color or show small light green or purple blocks that stayed stationary inside the window as you moved the primitive around - the whole look liked corrupted memory that stayed corrupted and was only visible when the primitive rendered over it.

It's a pity I didn't take any screenshots but I didn't think of that back then.

I was totally at a loss with this behavior, so I rebooted which didn't fix it. Somehow I was hoping that the gfx card broke in the last month and it wasn't my code which caused this totally inexplicable behavior. To determine this I logged into another workstation in our lab, checked out my code and run the program. To my horror it had similar problems... not quite the same but still strangely colored sprinkles all over the editor window when the primitive was rendered there. Both workstations are using graphics cards from Nvidia (8800 GTX if I recall correctly).

I tried to determine the revision which broke rendering with a binary search like trial & error through our revision history at work and found out that it broke when I changed the way the primitives were rendered to the texture.

It used to simply iterate through all primitives and render them once into the FBO.

The paper I'm implementing at the moment required a special solution for overlapping primitives though, so I switched to a more complicated 2.5 pass rendering.

If $c_i$ is the color value of the ith primitive and $\alpha_i$ its alpha value, then the resulting color and alpha are calculated as follows: \[\left< c, \alpha \right> = \left< \frac{\displaystyle\sum_i^n{c_i \alpha_i}}{\displaystyle\sum_{i}^n{\alpha_i}},\max_{i=1..n}{\alpha_i} \right>\].

For this I do three passes and use two floating point textures:

Render the primitives into a temp texture with
```
   glBlendFuncSeparate( GL_SRC_ALPHA, GL_ONE, GL_ONE, GL_ONE );
```
to get \[\left< {\displaystyle\sum_i^n{c_i \alpha_i}},{\displaystyle\sum_{i}^n{\alpha_i}} \right>\]
Next I set the FBO to the real transfer function texture and use the temp texture as source and divide the color values by their alpha using a custom shader.
Finally I use
```
  glBlendEquation( GL_MAX );
```

and only masks all except the alpha channel and render all primitives again into the transfer function texture. This gets the correct term into the alpha channel of the texture.

This usually works and although it's a bit cumbersome it yields the correct result. Only now I identified it to be the source of all my trouble.

Here's the relevant part in our source code:

_checkGLErrors();
m_fbo.AttachTexture( GL_COLOR_ATTACHMENT0_EXT, m_tempTexture.glTarget, m_tempTexture.glTexID );
assert( m_fbo.IsValid() );
m_fbo.Bind();
_checkGLErrors();

glClearColor( 0.0, 0.0, 0.0, 0.0 );
glClear( GL_COLOR_BUFFER_BIT );

// C = srcC * srcA + dstC = C * A + C' * A' + ...
// A = srcA + dstA = A + A' + A'' + ...
glBlendFuncSeparate( GL_SRC_ALPHA, GL_ONE, GL_ONE, GL_ONE );
glBlendEquationSeparate( GL_FUNC_ADD, GL_FUNC_ADD );

for( Primitive::PList::const_iterator i = m_primitives.begin() ; i != m_primitives.end() ; i++ ) {
    (*i)->bake();
}

m_fbo.Unattach( GL_COLOR_ATTACHMENT0_EXT );
m_fbo.Disable();
_checkGLErrors();

// divide color by alpha
glMatrixMode( GL_PROJECTION );
glLoadIdentity();

glBlendFunc( GL_ONE, GL_ZERO );

procInf.imageOp( m_tempTexture, m_transferFunction, m_program );

// render again this time to figure out the max alpha value

glMatrixMode( GL_PROJECTION );
glLoadIdentity();
glOrtho( 0.0, 1.0, 0.0, 1.0, -1.0, 1.0 );

m_fbo.AttachTexture( GL_COLOR_ATTACHMENT0_EXT, m_transferFunction.glTarget, m_transferFunction.glTexID );
assert( m_fbo.IsValid() );
m_fbo.Bind();
_checkGLErrors();

glColorMask( GL_FALSE, GL_FALSE, GL_FALSE, GL_TRUE );
glBlendEquation( GL_MAX );

for( Primitive::PList::const_iterator i = m_primitives.begin() ; i != m_primitives.end() ; i++ ) {
    (*i)->bake();
}

Now I was ready to hunt the bug but the problem the revision was written more than a few days before I left in August and it was working back then. I refused to believe that it was broken and I went to my boss who had been using his rig in the last days and indeed it worked. I couldn't really believe my eyes. Two workstations appeared to be broken for some reason (driver issues or even the gfx cards? - who would know..).

I went back and somehow managed to change the appearance of the bug on one of the workstations - all the framebuffer corruptions went away and were replaced with what seemed like an unchanging alpha channel of one of the FBO textures with alpha being 0 on the initial position of the primitive and 1.0 everywhere else, so it masked it out at its initial position.

For some reason I thought that maybe using BlendFuncSeparate and the normal BlendFunc function together maybe left OGL's state machine in a mess for some reason so I added calls to disable and enable blending to code to render the call to glBlendFunc useless (see the code above and/or below).

_checkGLErrors();
m_fbo.AttachTexture( GL_COLOR_ATTACHMENT0_EXT, m_tempTexture.glTarget, m_tempTexture.glTexID );
assert( m_fbo.IsValid() );
m_fbo.Bind();
_checkGLErrors();

glClearColor( 0.0, 0.0, 0.0, 0.0 );
glClear( GL_COLOR_BUFFER_BIT );

// C = srcC * srcA + dstC = C * A + C' * A' + ...
// A = srcA + dstA = A + A' + A'' + ...
glBlendFuncSeparate( GL_SRC_ALPHA, GL_ONE, GL_ONE, GL_ONE );
glBlendEquationSeparate( GL_FUNC_ADD, GL_FUNC_ADD );

for( Primitive::PList::const_iterator i = m_primitives.begin() ; i != m_primitives.end() ; i++ ) {
    (*i)->bake();
}

m_fbo.Unattach( GL_COLOR_ATTACHMENT0_EXT );
m_fbo.Disable();
_checkGLErrors();

// divide color by alpha
glMatrixMode( GL_PROJECTION );
glLoadIdentity();

---> glDisable(GL_BLEND); <---
glBlendFunc( GL_ONE, GL_ZERO );

procInf.imageOp( m_tempTexture, m_transferFunction, m_program );

// render again this time to figure out the max alpha value

glMatrixMode( GL_PROJECTION );
glLoadIdentity();
glOrtho( 0.0, 1.0, 0.0, 1.0, -1.0, 1.0 );

m_fbo.AttachTexture( GL_COLOR_ATTACHMENT0_EXT, m_transferFunction.glTarget, m_transferFunction.glTexID );
assert( m_fbo.IsValid() );
m_fbo.Bind();
_checkGLErrors();

---> glEnable( GL_BLEND ); <---
glColorMask( GL_FALSE, GL_FALSE, GL_FALSE, GL_TRUE );
glBlendEquation( GL_MAX );

for( Primitive::PList::const_iterator i = m_primitives.begin() ; i != m_primitives.end() ; i++ ) {
    (*i)->bake();
}

m_fbo.Disable();

When I ran the program, it decided to work normally and render everything the way it should. I was stunned once again. Now the good thing with causality is that everything is supposed to be repeatable, so I removed the statements again and was expecting to see the bug appear again.. - but guess what it did not and suddenly the machine was working again as supposed.

I tried the same fix with the other workstation in our lab and it fixed it before I had time to take some screenshots of the persisting framebuffer corruption.

I'm still at a loss here and I guess I won't be able to come up with an explanation of my own for this, so if anyone has an idea, feel free to tell me :-)

Cheers,
Andreas

VSTorqueScript #4

Andreas 'BlackHC' Kirsch — Fri, 12 Sep 2008 00:46:05 +0000

My little pet project is actually called VSTorqueScript, so I've decided to use the correct title for my posts, too.

Today has been a somewhat productive day, although not as productive as I've hoped, because I was stuck with a very stupid issue for 3 or 4 hours. I'm not proud of it, but it made me go to the VSX forum and become active there.

I've worked more on the debug engine afterwards (well, actually I went jogging and swimming - in that order.. think about it) and made some progress. Quite a bit of the stub stuff is implemented now and I think I'll start implementing the real thing tomorrow.

No screenshots this time, I'm sorry, but for the SDK users, I have a little helper file. If you look at the managed debug example, you'll find an AD7Events file that wraps the debug event interfaces. I wasn't easily able to port it over, so I decided to write my own version of it (while keeping the comments).

I've appended it to the post, if you want to take a look.

On other news I'll probably switch this blog over to my own webspace soon to get some advantages (like expandable sourcecode amongst things), but I think I'll stick to wordpress.com for a few more weeks. It's really a pity that you can't redirect from wordpress to your domain, but only vice-versa :(

Anyway, off to work some more on my code.

Cheers, Andreas

Here it is:

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using Microsoft.VisualStudio.Debugger.Interop;
    using Microsoft.VisualStudio;
    
    namespace BlackHC.VSTorqueScript.DebugEngine
    {
        #region Event base classes
    
        interface IEvent : IDebugEvent2
        {
            uint Attributes
            {
                get;
            }
    
            Guid IID
            {
                get;
            }
        }
    
        abstract class Event<ieventtype> : IEvent where IEventType : class
        {
            public Guid IID
            {
                get { return typeof(IEventType).GUID; }
            }
    
            public int GetAttributes(out uint eventAttributes)
            {
                eventAttributes = Attributes;
                return VSConstants.S_OK;
            }
    
            public abstract uint Attributes
            {
                get;
            }
        }
    
        class AsynchronousEvent<ieventtype> : Event<ieventtype> where IEventType : class
        {
            public override uint Attributes
            {
                get { return (uint)enum_EVENTATTRIBUTES.EVENT_ASYNCHRONOUS; }
            }
        }
    
        class SynchronousEvent<ieventtype> : Event<ieventtype> where IEventType : class
        {
            public override uint Attributes
            {
                get { return (uint)enum_EVENTATTRIBUTES.EVENT_SYNCHRONOUS; }
            }
        }
    
        class StoppingEvent<ieventtype> : Event<ieventtype> where IEventType : class
        {
            public override uint Attributes
            {
                get { return (uint)enum_EVENTATTRIBUTES.EVENT_ASYNC_STOP; }
            }
        }
    
        #endregion
    
        #region Event classes
    
        /// <summary>
        /// The debug engine (DE) sends this interface to the session debug manager (SDM) when an instance of the DE is created.
        /// </summary>
        sealed class EngineCreateEvent : AsynchronousEvent<idebugenginecreateevent2>, IDebugEngineCreateEvent2
        {
            private IDebugEngine2 engine;
    
            public EngineCreateEvent(IDebugEngine2 engine)
            {
                this.engine = engine;
            }
    
            #region IDebugEngineCreateEvent2 Members
    
            public int GetEngine(out IDebugEngine2 pEngine)
            {
                pEngine = engine;
                return VSConstants.S_OK;
            }
    
            #endregion
        }
    
        /// <summary>
        /// This interface is sent by the debug engine (DE) to the session debug manager (SDM) when a program is attached to.
        /// </summary>
        sealed class ProgramCreateEvent : AsynchronousEvent<idebugprogramcreateevent2>, IDebugProgramCreateEvent2
        {
        }
    
        /// <summary>
        /// This interface is sent by the debug engine (DE) to the session debug manager (SDM) when a module is loaded or unloaded.
        /// </summary>
        sealed class ThreadCreateEvent : AsynchronousEvent<idebugthreadcreateevent2>, IDebugThreadCreateEvent2
        {
        }
    
        /// <summary>
        /// This interface is sent by the debug engine (DE) to the session debug manager (SDM) when a program is loaded, but before any code is executed.
        /// </summary>
        sealed class LoadCompleteEvent : StoppingEvent<idebugloadcompleteevent2>, IDebugLoadCompleteEvent2
        {
        }
    
        /// <summary>
        /// This interface is sent by the debug engine (DE) to the session debug manager (SDM) when a thread has exited.
        /// </summary>
        sealed class ThreadDestroyEvent : StoppingEvent<idebugthreaddestroyevent2>, IDebugThreadDestroyEvent2
        {
            private uint exitCode;
    
            public ThreadDestroyEvent(uint exitCode)
            {
                this.exitCode = exitCode;
            }
    
            #region IDebugThreadDestroyEvent2 Members
    
            public int GetExitCode(out uint pdwExit)
            {
                pdwExit = exitCode;
                return VSConstants.S_OK;
            }
    
            #endregion
        }
    
        /// <summary>
        /// This interface is sent by the debug engine (DE) to the session debug manager (SDM) when a program has run to completion
        /// or is otherwise destroyed.
        /// </summary>
        sealed class ProgramDestroyEvent : StoppingEvent<idebugprogramdestroyevent2>, IDebugProgramDestroyEvent2
        {
            private uint exitCode;
    
            public ProgramDestroyEvent(uint exitCode)
            {
                this.exitCode = exitCode;
            }
    
            #region IDebugProgramDestroyEvent2 Members
    
            public int GetExitCode(out uint pdwExit)
            {
                pdwExit = exitCode;
                return VSConstants.S_OK;
            }
    
            #endregion
        }
    
        #endregion
    
        /// <summary>
        /// Extensions for IDebugEventCallback2 implementations
        /// </summary>
        static class IDebugEventCallback2Extension
        {
            static internal int Event(this IDebugEventCallback2 pCallback, IDebugEngine2 pEngine, IDebugProgram2 pProgram, IDebugThread2 pThread, IEvent pEvent)
            {
                Guid iid = pEvent.IID;
                return pCallback.Event(pEngine, null, pProgram, pThread, pEvent, ref iid, pEvent.Attributes);
            }
        }
    }

Visual TorqueScript #3.75

Andreas 'BlackHC' Kirsch — Wed, 10 Sep 2008 10:55:44 +0000

Another minor update, which I'm mostly using to publish a useful patch. I've forgotten to mention that I've already created a custom settings page for my Visual TorqueScript projects. Yesterday night (before going to bed) I decided to add a configuration dependent page, too. As you'll see it's very Torsion-esque, which is intentional and I plan to support .torsion files directly (as project files) later-on.

The main reason is that Torsion rocks and there is no reason to have yet another project file format for TorqueScript projects.

I hope that it will also be an incentive for people to give my plugin a try (if I ever release it).

A few notes (programatically so to speak :D):

Again the managed project framework is awesome and once you figure out where to look for things you'll certainly find eternal bliss, too..
I generally derive from SettingsPage directly, because I don't want any of the regular faff in my project.
Don't forget to publish your pages via ProvideObject(or ProvideClass from my previous post)
You don't need to create your own GUIDs for everything. Classses are automatically provided with a unique GUID - try it and access typeof(SomeClassWithoutGuidAttribute).GUID and it'll still work.
(If I said something wrong, please correct me..)

SettingsPage has an IsDirty property which you have to set, when a property is changed (to update the Apply button state). It results in pretty ugly copy'n'paste code, because you can't use default property accessors anymore. To fix this I spent an hour or two and I've come up with an elegant (in my opinion) patch to fix the issue:

Index: ProjectBase/SettingsPage.cs
===================================================================
--- ProjectBase/SettingsPage.cs (revision 10)
+++ ProjectBase/SettingsPage.cs (working copy)
@@ -27,7 +27,26 @@

 namespace Microsoft.VisualStudio.Package
 {
+    // Added to make it easier to edit pages [9/10/2008 Andreas]

+    class SettingsPagePropertyDescriptor : DesignPropertyDescriptor
+    {
+        public SettingsPagePropertyDescriptor(PropertyDescriptor prop)
+            : base(prop)
+        {
+        }
+
+        public override void SetValue(object component, object value)
+        {
+            if (component is SettingsPage)
+            {
+                SettingsPage page = (SettingsPage)component;
+                page.IsDirty = true;
+            }
+            base.SetValue(component, value);
+        }
+    }
+
     /// <summary>
     /// The base class for property pages.
     /// </summary>
@@ -78,7 +97,7 @@
             get { return this.grid; }
         }

-        protected bool IsDirty
+        protected internal bool IsDirty
         {
             get
             {
@@ -203,6 +222,13 @@

         #endregion

+        #region overriden methods.
+        public override DesignPropertyDescriptor CreateDesignPropertyDescriptor(PropertyDescriptor propertyDescriptor)
+        {
+            return new SettingsPagePropertyDescriptor(propertyDescriptor);
+        }
+        #endregion
+
         #region IPropertyPage methods.
         public virtual void Activate(IntPtr parent, RECT[] pRect, int bModal)
         {
@@ -410,6 +436,8 @@
                 try
                 {
                     Marshal.WriteIntPtr(ppUnk, p);
+                    // Reset IsDirty before rebinding all properties [9/10/2008 Andreas]
+                    IsDirty = false;
                     this.BindProperties();
                     // BUGBUG -- this is really bad casting a pointer to "int"...
                     this.grid.SetSelectedObjects(1, ppUnk.ToInt32());

It also contains a one-line fix to reset IsDirty when reloading the page or switching configurations (ie. any time the properties are reset to their old value).

Cheers,
Black

Visual TorqueScript #3.5

Andreas 'BlackHC' Kirsch — Tue, 09 Sep 2008 22:52:30 +0000

Small update: After hours and hours of error and trial (and implementing a few interfaces by using Copy&Paste wisely), I've finally managed to get Visual Studio to load my debug engine.

This looks gorgeous, doesn't it?

A few notes for those interested in writing Debug Engines:

See my previous post for the ProvideDebugEngine helper attribute
There exists a really nice managed debug engine example, that isn't part of the SDK. You can find it here.
You are required to set a PortSupplier, even if the documentation says otherwise (this was a major source of frustration in the last hour).

Since _ProvideDebugEngine_requires a Type for PortSupplier, I've been using this stub class:

    /// <summary>
    /// The class is just a stub for use with ProvideDebugEngine.
    /// It seems it's mandatory to provide a PortSupplier GUID with a debug engine.
    /// </summary>
    [Guid("708C1ECA-FF48-11D2-904F-00C04FA302A1")]
    internal sealed class DefaultPortSupplier
    {
    }

I've also added another attribute (ProvideClass) for convenience to register a class directly (it's the same as using ProvideObject on the current class type).

Since the code is quite short, I'm going to provide it here directly:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Microsoft.VisualStudio.Shell
{
    /// <summary>
    /// This is just a wrapper for ProvideObject to provide the class the attribute is used on
    /// </summary>
    [AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = true)]
    class ProvideClass : RegistrationAttribute
    {
        public override void Register(RegistrationAttribute.RegistrationContext context)
        {
            ProvideObjectAttribute objectProvider = new ProvideObjectAttribute(context.ComponentType);
            objectProvider.Register(context);
        }

        public override void Unregister(RegistrationAttribute.RegistrationContext context)
        {
            ProvideObjectAttribute objectProvider = new ProvideObjectAttribute(context.ComponentType);
            objectProvider.Unregister(context);
        }
    }
}

Cheers,
Andreas

Visual TorqueScript #3

Andreas 'BlackHC' Kirsch — Tue, 09 Sep 2008 16:24:55 +0000

I've spent a part of the last two days cursing the managed project framework (it's a bliss, too, and I'm thankful that it's available at the same time), because it is too different from what I need, and I'll probably end up rewriting most of it to use different approaches and not use _MSBuild _anymore.

I've spent more time, however, on reading up on Debug Engines in Visual Studio 2008 and I've started work on my own yesterday. To be honest, I haven't really started to work on it, I've rather spent the evening and most of today figuring out how to register a debug engine at all and write attributes for that, so that it becomes an automated process, just like it's done for Packages and Language Services by using attributes like ProvidePackage and ProvideLanguageService.

The code has turned out pretty nicely and now you can use the attributes like this, for example, to register a debug engine:

    [ProvideDebugEngine("{5887F58F-3B7F-401d-95CE-A7BADA1E4D7D}", "TorqueScript Debug Engine",
        Attach = true, ProgramProvider = typeof(VSTorqueScript_DebugEngine))]
    [ProvideIncompatibleEngineInfo( "{5887F58F-3B7F-401d-95CE-A7BADA1E4D7A}" )]
    [ProvideIncompatibleEngineInfo( "{5887F58F-3B7F-401d-95CE-A7BADA1E4D7F}" )]
    [ProvideAutoSelectIncompatibleEngineInfo( "{5887F58F-3B7F-401d-95CE-A7BADA1E4D7F}" )]
    [ProvideObject(VSTorqueScript_DebugEngine)
    [Guid(GuidList.guidVSTorqueScript_DebugEngineString)]
    public sealed class VSTorqueScript_DebugEngine
...

In my opinion this looks pretty neat - and it works.. / I really love the whole concept of attributes and the way they make coding a lot easier (even though it's hard to find information about useful attributes, as you only find out they exist when you stumble over them or write them on your own).

You can find the code here: http://www.icculus.org/~black/weblog/VS_DebugEngine/

A few notes:

The implementation was done making heavy use of copy&paste and quick replace (I copied the table containing the help description of the metrics into the source file and reworked it using RE magic to look the way it does now)
At first I used Type.GetProperties to enumerate all properties and write them out into the registry but that bloated the resulting registry entry, of course. Because of this I later switched to the HashTable approach that I first found in SDK's implementation of ProvideLanguageServiceAttribute.
The restriction that all attribute parameters must be constant expressions forced me to make a few changes - at first I had Guid properties and for the incompatible engines list I expected a Guid[], which didn't work. I changed the Guids to Strings and added the Provide*EngineInfo attributes instead of the incompatible engines list (same for the autoselect one, of course).
The Provide*EngineInfo attributes contained their registry code at first. This was ugly because they kind of required a ProvideDebugEngine attribute to exist, too, which I couldn't enforce. Later I changed them to be pure value holders and moved all registry code into ProvideDebugEngine.

Now nothing happens if you don't supply a ProvideDebugEngine attribute, too.

Now I'll try to start to work on the actual Debug Engine for TorqueScript.

Cheers,
Andreas

Visual TorqueScript #2

Andreas 'BlackHC' Kirsch — Sun, 07 Sep 2008 00:40:21 +0000

I haven't found time to add IntelliSense to the language service yet, so I've only added a basic parser and enabled brace highlighting/matching which was easier to do.

I've been told that a working debugger would be the most important feature, so I try to get started with it, but before I can do that I have to add a custom project type to support the standard .cs extension for TorqueScript and to offer a specialized debug options property page for Torque's telnet debugger.

Here you can see the progress I've made so far: I've added a new custom project type and changed everything to support TorqueScript files that use the usual .cs extension. There is no conflict with C# whatsoever. (At least from what I know..)

Next I'll look into removing the MSBuild faff (if possible at all) and adding the debug configuration page.

Last but not least I've found another neat tool that I can only recommend and want to share with whoever might read this: It's called VSSDK Assist and it's a so-called Guidance Package - it provides you with a neat assistance window that helps you do common tasks using the Visual Studio SDK in the preferred way (it supplies you with lots of dialog-based wizards for all kinds of things like custom pages/tools/editors/projects/...).

You can grab it here.

More to come.

Cheers,
Black

Visual TorqueScript #1

Andreas 'BlackHC' Kirsch — Fri, 05 Sep 2008 12:00:05 +0000

I've spent most of this week learning as much about Visual Studio's extensibility features as possible and I've decided to start a tiny project to see how easy it is to add a new language service to Visual Studio.

My conclusion so far is: it's not easy and that obvious but it certainly is easier than writing your own IDE. There is also the advantage that you have access to a nice debugging environment if you manage to write your own debugger service.

Anyway, I've started to work on adding basic support for TorqueScript to Visual Studio and after much learning, I've finished implementing the colorizer for TorqueScript today:

As you see I have to use the extension .tscript for now, because as you know .cs is already reserved for C#. I'm going to fix this later by adding a TorqueScript project type that overrides this link locally for your TorqueScript projects (I could live with that).

Creating your own language services is pretty easy, if you are ok with using C# and ManagedBabel. You just have to supply the specific grammar files for the managed lexer and parser generators and you're pretty much done with the basic stuff.

The Visual Studio SDK is pretty neat (you can grab it from here if you want) and another neat thing no one has probably heard about is that Microsoft gives away the Visual Studio Shell for free - yeah for free. That means you can build your own custom tools that have the Visual Studio look and feel and you don't have to pay Microsoft a dime for it.

A good place to start in general if you're interested is the VS Extensibility Center.

Stay tuned for more features (syntax parsing and debugging featues are the next thing on my plan).

Cheers,
BlackHC

Halo: Combat Evolved

Andreas 'BlackHC' Kirsch — Tue, 02 Sep 2008 22:37:08 +0000

Yay, I've finally found time to play through Halo 1 for the PC. To be honest I didn't even know that I owned the game a few days ago, but when I looked through one of my drawers, I found it together with Supreme Commander and GTA: San Andreas \o/

To make playing it look like work or at least like something worthier than just playing, I've taken notes occasionally and here they are:

The vehicles are very easy to steer: they always automatically try to drive towards your current view direction and you only use forward and backward to accelerate and slow down. I was used to ETQW's steering system and it took me some time to get used to, but it's pretty ok for the game.
I like to use my rifle as club. For the first half of the game I had a really closed-combat fighting style: start shooting while running on and when you need to reload or simply are near enough, hit the enemies with your gun before jumping to cover. It was very effective against Grunts and against Elites, too, since my timing was right most of the times and I was there, when their shield broke down. The ability to throw grenades and shooting at the same time (pretty much) also increased combat speed a lot and made everything more fluid, because I was able to react to everything always in an effective way. The Flood was pretty scary at first, because I had to learn a totally different combat technique and after getting used to it during the first few hours, I took me a bit to keep my distance again. After getting the shotgun and surviving the Library level fighting against the Flood became fun or at worst slightly boring and annoying, though I still have to watch out to prevent myself from becoming surrounded. I think, it is a pretty neat idea to make the Flood monsters get up again after being shot for the first time. It facilitates being surrounded and increases the risk of being overrun.
Level design is pretty weird sometimes. And with weird I mean repetitive - just look at the Library level to see the beauty of copy'n'paste. The last level, where you have to drive the buggy through a weird obstacle course that makes no sense whatsoever is pretty awkward, too - the buggy is a bit to wide to be driven through everything elegantly using the controls and I got stuck all the time. Moreover the buggy is too slow for all the little ramps you can find and I was left wondering why they put them there (maybe the buggy's speed was changed after the level was designed?)
I don't like checkpoints. Especially when they're used to tune the game difficulty by spreading them lighter in later levels >_<

So far, so good. I like Halo a lot and although some levels seemed a bit stretched (Library again), playing it was fun. I don't know, what I'm going to play next, but I think that maybe I should spend some of my time coding again - nah, just joking, next I'm going to play Call of Duty or more Sam & Max...

Cheers,
Black

Mirror's Edge

Andreas 'BlackHC' Kirsch — Mon, 25 Aug 2008 21:59:30 +0000

Mirror's Edge is one awesome game. There is hardly any information out about the game yet and the official homepage is lacking but if you're interested, there is at least one good fansite (I haven't checked for others yet) and a few trailers that really make me want to see more (or work for DICE - you decide..).

Check out the trailers:

If you're interested in some additional thoughts, please continue and read on:

First I have to say that I love the trailers and really really can't wait to see more. I always wanted to see a game that takes on the Le-Parcour concept and transforms it into fun gameplay - see one of my GID entries, that sadly wasn't finished, or games like N for example.

What exactly characterizes this Le-Parcour feeling? For me it's movement-centered fast-paced gameplay that rewards you with fluent animations and a feeling that you _flow _through the level in one progressive move.

From the games I've played the Prince of Persia series (beginning with Sands of Time) has managed to convey this feeling quite well. Particularly the escape scenes from the Dahaka are excellent:

[youtube=http://www.youtube.com/watch?v=dy4Mi7LneeU]

Mirror's Edge reminds me a bit of a mix of Oni (because of the hand-to-hand combat) and Prince of Persia, although it still appears to be very different.

One of the things I like is that it has no HUD whatsoever (or at least a very limited one to avoid motion sickness), which really helps immersion and this minimalistic feature goes hand in hand with the very clean design of the city (look at their choice of colors) and creates a setting that can only be described as stylish.

I think, it's an important point to make the players feel like they're playing a work of art and thus try to play it that way: trying to find elegant ways through the level, not brute-force ones.

The city conveys this, too. To return to the choice of colors: notice how everything is held mainly in white and all other colors are full, saturated and warm ones. They are also used in a clear and concise way. There is no mixing of them or gradients. It really sets forth the mood of the game in that part of it.

The use of the 'Runner vision' as visual hint is also very non-intrusive and stylized, too. It again fits the general design style. Though I have to say, that I don't think that those hints are sufficient as visual cues. I'd get totally lost in the level of the last trailer (if it's not entirely linear and there is only way to traverse the level because you also gotta have dead-ends then - this is probably true for the roof-top level, too) and it's pretty frustrating if you reach one, since it breaks the flow of the game.

I'd guess adding some arrows, that fade in, would certainly help it, but I'm curious how they are going to fix that anyhow.

Another problem I really wonder about is: what happens if things go wrong?

Nothing's worse than making one wrong move or time a key press badly and suddenly you have to load your last savegame or quicksave and replay the last 3 minutes, because you have forgotten to save. Especially when it comes to arcade games or platformers that only allow checkpoints, they tend to suffer from this a lot, because it's not really fun to replay the same parts of level over and over again, even if you agree, that it's probably just your fault for having two left hands.

The main feature that helped Prince of Persia against becoming terribly annoying was the time rewind power of his dagger - that is: you can rewind the last ten seconds and begin again with whatever you were doing from there on). Mirror's Edge could suffer from the issue, if they don't add something similar. Imagine dying, because you didn't rebalance Faith - that's the main protagonist in the game BTW - correctly while walking over a bar. Now you gotta wait a few seconds for the game to reload an old save point and start anew. This totally breaks the flow of the game. All motions and actions are stopped and you're forced to wait. You lose control and that's bad.

They really need to tackle that issue and I'm curious, how it's going to be solved in Mirror's Edge.

They also have to fix the sounds. The current moaning of Faith now and then gets on your nerves after a short time, too - just listen to the trailer again and imagine hearing that sound every minute for a few hours. It's a pretty scary prospect, isn't it?

That said, Mirror's Edge is one of the games that I'll try to watch actively for new developments and press releases, because it certainly could become a really fun and entertaining game that brings some fresh ideas to the first-person genre.

Cheers,
Black

Random Notes

Andreas 'BlackHC' Kirsch — Sat, 16 Aug 2008 13:35:26 +0000

I can't come up with anything to write right now, so I've decided to publish some random notes :-)

Random notes:

I'm listening to Duffy - Mercy on thesixtyone right now - I'm kind of surprised to find it there to be honest.
Right now I try to read "Design Patterns" by the "Gang of Four". Reading it is kind of tedious though because it contains a lot of text for simple things, but I don't just want to skim over it either, so I constantly have to force myself to continue reading.
I've stumbled over some good links from going through the Wikipedia entries for some design patterns. One interesting one in particular is: http://c2.com/cgi/wiki?PatternBacklash Quote (so true):

I've gained the impression that DesignPatterns violate OnceAndOnlyOnce. If you need to use a design-pattern often enough for it to become a 'pattern', it should be refactored into a single language statement just like every other 'pattern' you refactor. If you can't refactor the design-pattern due to language limitations or other constraints, you'll end up apply the same pattern, by hand, repeatedly (thus the name 'pattern'). Of course, given a TuringComplete language you can potentially refactor anything, but beware GreenspunsTenthRuleOfProgramming and the overhead required to refactor design patterns in any generic manner. The need for DesignPatterns should be considered a LanguageSmell. A language that has a lot of DesignPatterns smells a great deal worse than one that requires only a few.

-- DesignPatternsAreMissingLanguageFeatures
I've installed Visual Studio 2008 because I had a) access to it b) I once read that it contained some stuff to simplify the addition of new languages to it. There is an SDK available and I recommend to anyone to check it out - especially the stuff about using the Visual Studio Shell for your own projects (it's neat-o). My only issue with it is that all my work project are still using Visual Studio 2005 and there is no way to open 2005 projects without converting them which sucks - I let you guess why :-)
There is an SDK available for Visual Studio 2005, too.
I'm looking into writing a language extension for QuakeC, though I'm not sure whether it is worth the trouble, because most people I know who still code in QuakeC use Linux **>_>'**
Doing the same for TorqueScript might be a good idea though, and I could probably even sell it - again I'm not sure whether coding it would be that rewarding.
I've been playing Aquaria a lot in the last days and I really enjoy it, although it's not the kind of game I usually play. It certainly is worth its 30 USD.
http://grammar.ccc.commnet.edu/grammar/adverbs.htm is an awesome site about English grammar!
The latest Linux-Hater is simply awesome - and funny: Bundle me this

I think that's enough for now and I'm going to back to reading/playing/listening to music.

Cheers,
Andreas

Olympics in the PRC '08

Andreas 'BlackHC' Kirsch — Sat, 09 Aug 2008 21:04:22 +0000

Human Rights, Executions and the Beijing Olympics 2008

Short Note: This totally sums up what I think of the Olympics in China this year..

And the strip is really funny :D

What I've been thinking about lately..

Andreas 'BlackHC' Kirsch — Tue, 29 Jul 2008 21:48:58 +0000

I've just finished watching Donnie Brasco with some friends and I really have to say that it's an awesome movie. Al Pacino and Johnny Depp really deliver an amazing performance. It's based on a real-life story which is worth a look, too - there are a few differences to the movie, though.

Lately I've been watching quite a few movies, not that I would have that much spare time but after an exam, I spoil myself with a movie now, and then and since I've had four exams in the last week you can guess how many movies I've watched. I've been busy like a bee this term which also accounts for the lack of new posts on this blog in the last months and I've pretty much finished the second and the fourth term in one /

This means that I will have more time in the next terms to work at university and earn some money. Moreover it's not just about the money but also about the projects I plan to work on, but more on this later. Right now I'm working for the chair for medical visualization and augmented reality and my only (and current) project so far has been about implementing a paper about "Importance-Driven Volume Rendering", where I'm currently and still stuck with work on the UI for the transfer functions (the data only consists of intensity values and since people prefer colors, some preprocessing has to be done). The bad thing is that I've totally forgotten everything else over it - I've spent like two weeks reading up on volume rendering and raycasting in general and even wrote a very simple raycaster in RenderMonkey, but I hardly remember any of it anymore now..

Most of the lectures that I attended this term were pretty interesting (Linear Algebra, Theoretical Computer Science, Computer Networks) but Algorithms and Data Structures was more on the incompetent side of things due to the lecturer not preparing for any of the lectures beforehand and an incredible amount of errors in his slides - our weekly schedule was as follows: 2 hours of new subject matter, 1 hour of discussing the corrections of the errors he introduced in the former 2 hours. In the end I only attended that one hour because it was more fun and I didn't get as confused. My main source for learning about algorithms thus was the book (Cormen et al). I've also attended Software Engineering and I'm still not sure about what to think of it, so I'll wait for the exam and maybe reread Code Complete for it to get some common sense about the general topics back again.

I must admit that university hasn't told me a lot of new things so far. Especially nothing that I couldn't have learned on my own just as well or even better in less time to be really honest.

So all in all I must say that anyone who is bright (I consider that myself) and already knows a lot about programming is well-advised to try and get a job in the industry directly if possible because anything else is just a waste of time.

On other news I've finally acquired my own domain: blackhc.net after blackhc.com was already taken. I haven't really had any time so far to set it up properly, but I've already routed blog.blackhc.net to this blog, so it's not a total waste.. :)

I might also move this blog to it but I'm not entirely sure yet, because I'm pretty happy with wordpress.com's service so far. There is a little bit of irony to it, too, since I only got the idea of getting my own webspace, because I wanted to post a blog entry about matrices, etc. and needed formula support for it. I searched with google and everything I found required its own wordpress installation with plugins and so on. Then I started to look at domain names and found out that blackhc.com was already taken by someone else and that HostMonster was really very cheap (5 USD a month if you subscribe for 2 years) and the deal was pretty much settled. After I bought the domain and the hosting package, I went on to search how to install the plugins in wordpress and eventually found this FAQ posting.. Now I know that wordpress.com can do LaTeX after all and I have lots of cheap webspace, too - I should call that a net gain, right?

That said, I think this post has become long enough and I'll drop by in a few days and write more after my last exam. I also still have that linear algebra post in the queue and possibly a few others, so hopefully I'll have enough time to finalize them and publish them soon.

Cheers,
Black

Best of Cyanide & Happiness

Andreas 'BlackHC' Kirsch — Sun, 11 May 2008 13:43:50 +0000

Long ago (about three weeks ago to be correct) I still had lots of free time and no university lectures yet, so I decided to look at all - I repeat: at all 1000-something - Cyanide & Happiness comic strips on www.explosm.net and make a collection of the ones that I liked the most. Of course my collection isn't complete and I got pretty tired at the end but I still think I've found quite a few gems among the strips.

I've stored them as a list of links in a raw txt file (it can be found here, if you want take a look a it) and it wasn't really usable the way it was. I wanted to put it on my blog though to let others view it in a comfortable way but since I didn't have enough time I postponed it... till yesterday. Yesterday I've finally found enough time to sit down and think and code something nice to let you (and myself, too) browse through my favorites.

You can find the final page here: http://data.blog.blackhc.net/explosm/explosmviewer.html

How does it work?

It uses AJAX! Well actually it only uses XMLHttpRequest to query the .dat file that stores a preprocessed list of C&H links (plus the respective image links) and uses that data to let you browse my favorites dynamically but it contains the code to create the said .dat file out of my raw .txt file.

Consequently if I ever update my list, I can do so simply by updating the .txt file with a few links and regenerate the .dat file with all additional information.

I'm mostly using some RegExp to parse the Explosm pages and my .txt and .dat files. If you want to try the code out yourself, mind that you need to uncomment the compileList( true ); line and use Internet Explorer to run it because FireFox doesn't allow cross-domain access which is naturally needed if you want to parse the pages on explosm.net. The result is copied into the clipboard and can be then saved and uploaded to the webspace of choice (if you adapt the links in the script, too).

Enjoy :-)

Cheers,
Andreas

New Room, New Uni Term

Andreas 'BlackHC' Kirsch — Sat, 19 Apr 2008 20:24:08 +0000

Finally the second university term has started and I'm back in Munich working hard to get through university as fast as possible.

I've also moved into a new room - I'm staying at a dorm now - and it's really pretty awesome. There are lots of exchange students (Belgium, France, Italy, Japan to name a few) and we're all having a good time constantly, so really it's a huge improvement when it comes to meeting other people. On the other hands it not a typical dorm either: my room has got its own little bathroom and it's all new because the building has been renovated in the past year - although to be honest, some parts are still being renovated, which sucks because there is still a scaffolding outside my window and I can't really look outside.

But pictures can convey a lot more than words, so here are some low-res photos that I have taken with my web-cam.

SplashDamage Photos on Flickr

Andreas 'BlackHC' Kirsch — Wed, 19 Mar 2008 00:13:16 +0000

Do you know what a games company looks like? And if you think you do, let me add some adjectives that will change everything: an awesome games company, a cool games company, a SplashDamage-esque game company :D

We (some folks from the #q3mods) had a meeting recently at Arnout's, Gordon's and David's place in March - it was like from Friends, the series, only they have got a house, a gaming TV, lots of game and Guitar Hero 3 / They were so kind to let me stay in the dungeon (the basement) - for a week and I spent all of my free time playing Guitar Hero or watching them work at SplashDamage's office, which was grand. I can't really say a lot more, otherwise I'd get torn to pieces by some attorneys for violating the NDA I've signed but let me repeat: It was awesome. We all stayed at the their place, played games (Guitar Hero!! and lots of Wii games), watched movies, ordered pizza and Chinese and even ate it all up, too, and had a splendid time.

Cheers,
Andreas

Edu2 Postmortem

Andreas 'BlackHC' Kirsch — Tue, 18 Mar 2008 01:13:50 +0000

I think I've been talking about Edu2 a lot already and I don't really feel like giving a huge overview again - probably I will end up doing so nonetheless. Edu2 is finished, over, done. I'm not going to touch it again - ever! That said, I think there are a few nice things and lots of bad things to say and tell about it. I usually paint bleak pictures and I won't stop here, because I try to be honest with the negative side of things.

I guess this is going to be a bigger postmortem because there are many different topics that I should mention to give the reader a fair impression.

To begin with, a few links first for those interested in Edu1 and Edu2:

Interesting fact: google for Edu2 and you'll find a project that is almost about the same things that we worked on - except it's 2D only.

First let me tell you what Edu2 is about: Imagine Second Life, Facebook, some uni/educational content and mini-games (flash stuff) merged to one product. That is Edu2 from what I could figure out of the business plan and my work on the prototype.First observation: this sounds like a huge project - too huge for one university term. Thus we only worked on a 'prototype'. Second observation: it sounds like a project that is a tad bit megalomaniac and doomed to fail because the market is kind of saturated, too. I, honestly, think that this will be its likely fate, if anyone should ever try to develop it.

The project was actually about creating a business plan (half of the team) and creating a prototype (the other half), so I don't think it's a bad idea, because social networks really are big nowadays and experience doing social network stuff (even if it's only a very raw prototype) can come in handy maybe. It's mostly useless if you think that it's helping you get a job in the game industry because it's a very different business area and at most the prototype team may have learnt something valuable because it was working with mostly standard game tools.

One really funny note here: We visited Ubisoft Germany and EA Germany and presented our project there. At the time being there, I was somewhat excited to get an opportunity to talk to some important people there. However, I wonder: why did we even go to visit them? What do they have to do with our project? Why would they be interested? Obviously, they are not - except maybe EA, which we all know, plans to have a go for total world domination with unoriginal, corporate games (this is supposed to change though, so thumbs up) - so I question the use of the whole visits.

Had we got a simple but interesting game, it might have been of more use and we could have possibly pulled of what I would call a "Portal" and actually present something convincing that would have been a true first playable prototype of a game. So the first big point that deserves mentioning is:

A good project needs a concise and down-to-earth vision behind it.

Edu2 certainly didn't have one, because no one really had an idea of what to do or rather everything was pretty blurry. We had a few docs with very rough but mostly too vague ideas and the problem was that decisions could hardly be made because, of course, you had the big real thing looming behind the prototype. In the end everybody or rather tiny groups of people who talked about it decided from case to case without lots of deeper thoughts behind it simply to get things done.

Communication in general was pretty bad

Really there was little communication between the different groups and I, at least, had very little contact to everyone else, so it was very hard to involve the team in decisions - since we were the coders, we pretty much had control over the core functionality, because our level designer didn't have that much freedom: their task was to extend and clean up the original Edu1 campus level (a level consisting of a virtual university campus) and add gecko terminals now and then. We had a few fun ideas the night before the presentation, for example:

Adding zones that would open a gecko window with specific content - e.g. Amazon in the store or the mini-game selection window when you are near a gaming console..

Those were ideas that actually were really interesting and could have added a lot to Edu2's 'gameplay' but sadly time was a very constraining factor and in the end they didn't make it in, even though the code was pretty much in place. Additional trigger brushes would have been needed, but there simply wasn't enough time to recompile the map. Clearly, we could have come up with those things earlier if we had talked and discussed things more.

The only person I was talking a lot to was Frank 'res(2k)' Richter, a pretty damn good coder, who is currently working on the free and awesome Blender/CrystalSpace 'Apricot' game in Amsterdam. He did an amazing job with offscreengecko, a gecko client that makes it really easy to include gecko (XUL) in games, and the Edu2/DarkPlaces code we worked on in general - DarkPlaces is the name of the engine for newbies to this blog.

We talked a lot about things and it was great to have someone on the project, who is at least as experienced as I was and quite certainly a lot more experienced than I am to be honest. A few things went wrong there nonetheless:

I was chosen to be the coding lead between the two of us. Since there wasn't anyone else, I don't think, I really had much to do as "lead" and that there was a huge difference in what we did, but I tried to take responsibility and keep bugging him to work on stuff, question him about things that we needed to work on, try to decide on the tasks we should do next, etc.. It was a team process and I hope that he isn't too angry with me about it :-/
Again and again and this time, too, I've been way too protective of my code and way too restrictive and critical of other's code. When someone changes things in my code - that is code I've been working on - and I consider the changes to mess up the code (design) or be problematic - that can drive me nuts pretty quickly. So I guess I'm quite bad tempered when it comes to coding. I'm pretty glad that I only frustrated Frank once for real and I'm still very sorry about it and I hope that I've learnt my lesson now.
Even though I've tried my best, work was sometimes distributed awkwardly resulting in conflicts and things being done twice - although those few times I don't think that time was really totally wasted but it certainly wasn't used as efficiently as possible, which is bad because we both had other, more important things to do.

Another fun fact: no one has told our self-taught mappers and modelers how to do so effectively. Edu2's only map apparently had compile times of 20+ hours at the end of the project and we had lots of bloated models with ridiculous triangle and vertex counts without any real justification for it.

I'm not sure I should mention this, but we had a lot of trouble with our only character modeler Jarno Müller, who, to put it in a nut-shell, wasn't a least bit as professional as he claimed to be - considering the fact that he seems to own his own company that does gfx contract work. I've already ranted about him in a past blog entry, but I think he deserves to be named, because otherwise the way he behaved and partially crippled the project - we had to use standard free Quake3 models in the end for god's sake - wouldn't make a difference and the incredibly unprofessional way he acted and left the project (by simply not replying to emails at all in general anymore and not participating seriously in any way before either) pissed me (and others) off big time. He was offered help multiple times in the beginning to try and see how to export models from Cinema4D into a format Darkplaces supports but didn't answer any emails and then a week or two after we extended his deadline to produce one model at least he sent us a model we couldn't open because he used the latest C4D version and there was no converter to be found quickly - and he, of course, demanded that we tell him how to export the model to DP and he didn't really adhere to one requirement that he was told from the very beginning (Quake3 multi-part model, easy support for different skins, multiple low-poly models - WoW-style).

Anyway, to sum everything up, here's an overview of some of the pros and cons:

Pros
- I have meet some interesting people and also made a few new friends (hopefully).
- I finally worked again on DarkPlaces, which was nice, because I have only been working with GarageGames engines for the past two years.
- We pretty much started from scratch game-code wise and had Edu2 actually been a real game and had there been a FPP in the end, it would have been an ueber-useful experience. But even without that, it was pretty interesting.
Cons:
- I really have lost a lot of time with the project that I really could and should have used for other things.
- The final prototype really is mostly useless, except if I get myself motivated enough to port back some of its features to Nexuiz..
- The prototype hasn't been actually officially used for anything else than a small video.
- Thus I can conclude that it was a terrible waste of time
Worth mentioning:
- The crunch time in the last two days was really fun and sleeping ten hours in three days says it all.. It was probably the most fun I've had in a long time (before visiting SplashDamage again).

I think that's it for me.

Cheers, Andreas

Windows Weirdness

Andreas 'BlackHC' Kirsch — Sun, 16 Mar 2008 00:14:35 +0000

Lately I've spent a lot of time developing batch files for Edu2, writing for weirdness for batch renames and having fun with the command-line in general, since it is my tool of choice that goes beyond simple Explorer and SVN Tortoise jobs. And because of that I've found a few quirks and bugs that are either funny or at least worth noting since they are a PITA to track down.

How to crash cmd.exe

Plain and simple:

for /R %f in () do echo

crashes cmd.exe in Vista

Environment Variable Mess-Up

Maybe you know the environment settings dialog in XP/Vista. It allows you to specify environment variables both for the global environment and for the current user. (The former only if you're an admin though.) A neat feature is that you can, of course, set the same environment variable twice and Vista/XP will merge them to one, which naturally allows for some neat system configuration.

However, it has its issues, too, of course.. Notably it handles case-sensitivity of environment variable names inconsistently, which results in some pretty weird behavior.

For example let's assume we have a global and a user PATH environment variable, then Windows would merge the two, so that for applications PATH will be %PATH_global% %PATH_user% (where _global and _user refer to specific variables). You can easily try that by opening cmd.exe and typing in set path and notice how they are merged.

Now what happens if you have e.g. a global Path and a local PATH. You'd expect that they either show up as two different environment variables (if Windows treats them as case-sensitive) or they will be merged into one variable again (if they are treated case-insensitively). Sadly none of the above happens: $ set path will return Path = %Path_global% %Path_global%, which is both useless and can cause severe headaches, when only one half of your environment variable is available suddenly (for example after installing a program that changes or adds environment variables).

What happens? My guess and the only possible explanation are that Vista's (maybe XP's, too) handling is inconsistent. Detection of variables that need to be merged is case-insensitive, while the actual merging is not. It fails to find the user environment variable and instead uses the global one again (maybe it uses the same buffer to cache their value before concatenating them).

It took me some time to figure it out because, obviously, the set command and everything else treats them as case-insensitive.

File Associations

I've been using Notepad++ for over half a year now and I really really like it. But a few days ago I made the mistake of switching over to using it exclusively over notepad and for that I decided to change its file associations. Thank god it provides a simple dialog for that in its preferences window and it was a nice and easy task. Except that I also changed the file associations for .bat and .cmd files and instead of changing the "Edit" verb it changed the "Open/Execute" verb, which caused .bat files to only be opened in Notepad++ instead of being executed.

Windows 95/98/ME/2000/XP had a really nice file association tab in their folder options which gave you total control over pretty much all aspects (except for some obscure cases where you had no choice but use regedit to change things). Now Vista's designer apparently have decided that it gives users too much control and that that is a bad idea - so they have totally crippled it. I repeat: totally and fully crippled it. Instead of being able to edit and manage various verbs, specify DDE actions, setting a default verb, etc. etc., you only have a "Change Action" button that only allows you to specify the command-line. They have also moved it to System Settings, which is probably a better location than Folder Options where it was next to the visual appearance tab, but it's totally useless now - as are more freeware tools that you find on the internet :(. After some cursing, goggling and worrying I eventually found some .reg files that restore Vista's default associations but that's really not what I would call a perfect solution.

Stay tuned for a last blog post about Edu2: a post-mortem (coming soon hopefully) and more,
Andreas

Edu2 is finally done \o/

Andreas 'BlackHC' Kirsch — Tue, 04 Mar 2008 21:57:13 +0000

Edu2 is finally over (at least for me). I've uploaded a 99% final build - the only thing that is missing are correct credits, but I hope to receive them ASAP and then I'm really really really done with it. Which means lots of free space on my harddisk - the beast is over 2 GB big here...

On the other hand lots of new awesome projects are out there waiting for me to contribute and I've already joined two new projects (WoWar and SemuruC) and also finally have time to concentrate on another one (with the mysterious name "DasProjekt"). More news, links and information about them later.

First I have to come up with a Post Mortem for Edu2.

Cheers,
Black

PS: On different other news I hope that I have been able to convince the head of PR of EA Germany to use Nexuiz as FPS example game at one of their public discussion rounds. More on that later, too - hopefully!

Final Java Homework at uni

Andreas 'BlackHC' Kirsch — Tue, 19 Feb 2008 22:44:58 +0000

This is the final Java homework in our Java introductory course at university. Enjoy :-)

Click here to play the game

Greetings,
Black

Edu2p Stuff

Andreas 'BlackHC' Kirsch — Sun, 17 Feb 2008 23:37:52 +0000

Finally, I've found some time to write a tiny post linking to a couple of pics of Edu2 (the prototype) - the project that has kept me busy for the last 3-4 months and burned me out pretty badly from what I can tell..

First a small animation that shows the terminal:

Working terminals in edu2

This was an awesome step in my opinion and it's really great to see it live (especially in a real level).

Nexuiz's Homepage in its (new) default menu

Res2k aka Frank Richter hacked that into Nexuiz's new menu (written by div0 - kudos to him for fixing/rewriting the broken piece of shit that my old menu was after all). It's pretty nice but of course Nexuiz doesn't support Gecko atm (and won't either in the near future due to potential security vulnerabilities), and that feature only existed in our branch for some time. I guess I'll add it back soon, too, because it was really useful.

Next comes Edu2's chat window which is really a big step forward because it uses client-side code and the menu to replace engine features (ie. the normal messagemode input line when you want to chat). This should be an example for more serious projects, too, because it was really easy and clean and it's really a lot more professional than the usual DarkPlaces Quake1 look.

The first version of the Edu2 chat window

Of course, it wasn't finished and still isn't fully, so I added a button to view a chat log of different sizes (apparently I don't have a screenshot of that and I'm too lazy to grab one so here it is without the varying chat log size feature):

Chat window with chat log

Next we spent some time adding model rendering support to the menu. Poor res2k spent a few days writing lots of code and doing many changes in a separate branch that got mostly thrown away by me later and/or rewritten due to bad management (that would be) but that prototype code was used as a base and was probably cleaner than the current result hiding in DP's codebase.

Anyway a nice first pic after the "hack":

Model rendering in the menu

And even with alpha-blending support - that required some "clever" thinking on my side - and maybe div0 will add that to Nexuiz at least because it was on his wish-list:

Alpha-Blending Support When Rendering Models

We also wanted to feature at least one mini-game (actually we wanted to design something proper but in the end it was our job again to do it and in the very end I volunteered due to having a similar task in my introductory computer science course at university). Guess the game:

Guessed? Yeah pure, good, simple ol' MineSweeper! It looks like that in its final version though:

Even div0 didn't think that such things (resizeable and moveable(!) windows and dynamic items) would be possible in his menu but I hacked that in pretty quickly and it's not even that big a hack to be honest :-)).

In the mean-time res2k and I have worked on the player model in the game itself - res2k did the groundwork and it's a very good one to say this and when he was away for a weekend I simply added animation support from Zymotic and adapted it to our needs somewhat- res2k was a bit angry with me when he came back and found animations somewhat working already but I let him- I didn't really have any spare time either - finish the rest of course and he also added some code to allow taunts to be sent from clients to server which broadcasts it to all clients which does pretty awesome things using temporary entities in QuakeC - which I'm pretty proud of coming up with to be sincere. Another big TODO which res2k worked on and which shall not go unmentioned here is all the gecko stuff - except for the dynamic texture code in DP and the first set of gecko builtins and wrappers, he created all of our Gecko backend - which is a free sourceforge project called OffscreenGecko - and even added support for JavaScript interaction with Darkplaces (the various VMs). A major task and because he's currently in the Amsterdam working on CrystalSpace and the like, it's up to me to kick some asses and use that in Edu2p for some new ueber-awesome features.

Anyway, what I've been working on lately is skin support in edu2p - thus another contribution to the playermodel code :P - and intergrating it with the menu plus allowing the users to select different models for the player's legs, torso and head. I think it looks pretty great with Quake3's models but sadly we can't use them and many of the free ones that can be found on ioquake3's homepage are incompatible and badly tagged (or rather Darkplaces has problems with them that Quake3 has not).

Now imagine that with more WoW-like models and you can see where Edu2(p) is heading. Sadly we have no player modeler right now, who could create such models and the modeler/animator we had that was capable of this was - I sadly have to conclude in retrospect - a complete jerk and arrogant retard - and yes that is mean and probably uncalled for in this public blog but I'm not naming him and I'm pretty pissed off that he still hasn't replied to 10 emails from me, hasn't done the work he was more than once politely asked to and agreed to, didn't give any feedback though we were more than willing to help and adjust requirements if needed and used way too many repeated exclamation and question marks for my taste in his emails and simply dropped of my radar. I'll still try to see if I can fix the models that we can use but there's a grim perspective to it with little more than two weeks till the final presentation.

Anyway this is it from me and stay tuned for a really final update on edu2p and more stories and rants and disclosures - no NDA! \o/

Black

About "Jugend denkt Zukunft"

Andreas 'BlackHC' Kirsch — Sat, 19 Jan 2008 00:57:25 +0000

About 1.5 years ago some students (including me) from our school participated in a federally supported project called "Jugend denkt Zukunft" ("Youth think Future" - and yes it sounds shitty in German, too, and uses bad grammar...).

Interestingly enough most articles that have been translated don't translate the grammar error, too, examples here, here and here.

Let me explain what it is about: The school partners with a local company (or rather the other way around because it seems that the local company has to pay a lot of money (some 6000 EUR) to the organization - NGO, of course - to participate and you (that is if you are the students) are part of a "role-playing game", in which you first learn about "mega trends" of the future and then in the last two days have to come up with new product ideas that have more or less (mostly less) to do with the company and its current product lines.

The problem is, if you have to come up with new products for a service-provider (computer-related would be paradise probably) or at least one that directly sells to end users, you're lucky, because you can actually come up with something useful or feasible (maybe). But if you end up at a company that produces screws, you're_, well,_ screwed.. because you'll never find something that is useful or makes sense and has something to do with your company in a way that won't leave people scratching their heads afterwards.

Another big point is that I pity the company that gets new ideas from a group of pupils that it couldn't come up with on its own - lock a few of your employees in a room for a week and you'll probably get lots of good ideas, too, - without having to pay 6000 bucks and you can still lay them off it they don't ;) - but it's all about PR and publicity and making yourself look good in the public eyes and less about being efficient and actually producing results or teaching the students anything except faking and presenting fake results.

But even worse they don't consider the effects on the pupils themselves. Everybody congratulates them for what they've done and how innovative their fake products are (the products are mostly based on concepts and technologies that are yet to be invented) and how great they are after all. It's like entering bizarro world if you're used to the real world with real research and real products. The worst moment I've experienced was when some otherwise cool students started to bitch about not being placed better in a state ranking of the best "Jugend denkt Zukunft" projects and in their opinion"they didn't get a prize because their product was too realistic and too innovative" - and I wanted to smack him over the head too badly but resisted in the last moment. Their "product" was an idea consisting of a couple of fake powerpoint presentations that contained no actual information what-so-ever and a wrapped book that ought to represent an eBook-like thing.

So actually all you learn from doing those projects is how to produce professional-looking bullshit. Not less but not more either. It's sad and pathetic and the PR crowd probably cheers for they have taught a new generation that you can quite probably sell shit if it's only gift-wrapped nicely enough.

My opinion might be a too pessimistic one but I've gone through the actual project week and felt pathetic, attended two nominations for the project a few months ago - first the state-wide nomination of the best projects in Bavaria (was sometime in October) and we were placed third - in your face, guy from above! - and then the federal nomination in Berlin (it took place last December) and we even won first place (with two other schools) and I still felt very pathetic and both times my main thought was "what the fuck? why are we rewarded anything for this fake crap?" (our school won some good money by the way).

Moreover the whole nominations were crap, too. They were meant to be "innovation fairs" (another crappy name, but I simply want to convey the bad wording of the German expressions they used) and the one in Bavaria was a run-away joke - a fair of 5 teams presenting 5 bad ideas and the winners were already chosen beforehand, so it didn't matter at all. And it was sponsored by Wrigleys - I must mention this because we're shown a Wrigleys' video and a Wrigleys representative was there and talked and the big nomination in Berlin wasn't better either. That one was sponsored by Vodafone and two other companies (sorry I didn't listen this time at all) and this time there were maybe 15 teams and it was possible to win an audience award - at least we were told.. In the end it turned out that the audience award really wasn't determined by the audience but by the groups themselves because in the final gathering before or after the sponsors talked each group's name and idea would be mentioned and the ones who clapped and cheered the loudest would win. I shouldn't mention they only had one microphone for measuring the noise level and it was in the front at the stage and as incredible as it may sound the group who sat in front - won - oh my god, what a surprise! But kudos to the group who came up with the whole fair scheme (except for the clapping idea it was mostly great) and also kudos to IFOK for out-sourcing that to students, too.

The secretary for families and youth even talked at the federal nomination and congratulated us for being a new elite and the innovators of the future in Germany. So instead of actually encouraging real research and real projects they make us participate in a project that will at best help us to obtain credits by false pretenses later in our lives... but of course, you can argue that you should foster both (actual research and presentations of it) but I'm still shocked by the opinions of some students who participated and simply think that they actually are great because of it.

Cheers,
Andreas

But now I've ranted enough about it (and I want to finally get it posted)

Nice Comparison of Free-Software Shooters

Andreas 'BlackHC' Kirsch — Mon, 31 Dec 2007 14:41:28 +0000

On http://www.linux-gamers.net/smartsection.item.81/comparison-of-free-software-shooters.html you can find a very nice comparison, which is pretty extensive and to my shame I have to admit that I haven't played many of them - I really ought to try War§ow soon.

Nexuiz gets a pretty good review (in your face, AlienArena :P) but still note:

While most of the game is cleaned up far beyond its Quake 1 roots, it is still lacking in presentation with the menu being very circa 1990s.

/me sighs

This Evening's Worklog

Andreas 'BlackHC' Kirsch — Tue, 25 Dec 2007 22:21:16 +0000

Today I've decided to work on BSS again - finally.

And I've even come up with a few neat bullet points to help me clean-up the codebase and I really feel like sharing them with the world today (although everybody can come up with them in a few minutes, too, but hey - now I can say that you've only copied my thoughts :-P):

How to fix and clean-up stuff quickly:

What do I want?

remove unnecessary code

clean-up the design

untangle tight couplings

How do I do that???

Hack & Polish

look at a specific code section

rewrite it quickly while keeping the behavior the same

don't look at the overall design concepts - to KISS (to keep it simple STUPID)_

as soon as the new code actually seems to work, go and polish it slightly (to make it readable again)

Repeat these steps a few times..

Now look at the "overall picture" and try to identify common patterns and remove duplicate code again by writing generic helper code - but this time keep it decoupled and make it less messy!

Rinse and repeat...

OK... sounds great? Yes, it does! And it even works to some degree. I've tried it this evening and although I - like always :-( - lose too much time browsing around and getting lost in the WWW, I still made some progress.

Here's my worklog as proof (and to tease Joshua a bit ;-P):

18:32	starting to work on BSS's cleanup
19:20	removed some Sploidz-specific code from the items module
20:00	removed more files, tracked down Joshua's new coder (wtf didnt he tell me?!?) and found out that two files were missing in the SVN repository (added them from a backup that I have with me here)
20:30	changed bits of EventFSM and t2dFSM
21:00	reworking BlockItem
21:45	BlockItem now uses a FSM
22:45	BlockGame now is decoupled from Grid (gamelogic-wise) and BlockItem from BaseItem (the latter might be reverted later though)
23:00	trying to rewrite BlockGame's setupItem function again but deciding that it's pretty late

Enough for today. One more Dexter episode for me now (actually the first today) and then good night!

Good night,
Black

Blender2md5 Exporter

Andreas 'BlackHC' Kirsch — Sun, 23 Dec 2007 22:44:52 +0000

Update: I've (sadly) worked on it some more today and there's a new version online. You can find it on icculus: http://svn.icculus.org/darkwar/trunk/base/scripts/

For the past 4 days I've been working on and off on some changes to der_ton's blender2md5 exporter - big thanks to him for developing the exporter in the first place - and in doing so I've finally learnt Python, which is an awesome scripting language to say the least.

I haven't really changed much of the exporter's internal workings (only a few clean-ups here and there which hopefully didn't break it) because I mainly concentrated on GUI changes. fleeky, a nice circus fellow who is DarkWar's only active modeller at the moment, pestered me long enough to add support for exporting multiple animations from Blender to md5, that I finally gave in and started working.

Thank God, Python's tutorial page is excellent, too - especially 'Learn Python in 10 Minutes', which really does a very good job at showing you many language features very fast - which arguably isn't useful for total newbies but it gives a nice overview and invites you to play around.. and is really helpful when you start to code because you already know about some "advanced" language features early-on and can look them up when you think that they might help you solve a problem more elegantly (in contrast to not knowing about the features at all).

Another tip for Python beginners: get ActivePython and Komodo Edit (or even the IDE if you have the money and/or are a student and willing to go through their registration process, which I might if I'll happen to work more with Python). Both are free and will get you started with Python coding quickly. They really helped me a lot while learning and coding the new features.

Anyway, so what did I code? First let's take a look at the original exporter interface:

Old blender2md5 Interface

My first changes were about adding a simple batch system that allows you to set the properties for one animation and mesh to export and then specify a batch file and append them to it. Then you can export all "jobs" from the specified batch file or directly specify it and export from it:

Interface with first change set

This was ok but it didn't feel that nice - since you'd have to manually edit the batch file if you wanted to change export properties, etc.

Today then I've decided to spend one last day on the project and rewrite most of the GUI: it now displays all default buttons at the bottom and allows one to insert as many animations to export as one wishes. It displays the animations each with their own target filename, start and end frame and action dropdown menu in a table and now also exports all other settings to settings files (batch files don't really describe it correctly). Even better I've added support for automatically saving the settings to Blender's .blend files which should really help automate exporting a lot.

Just open the scene file, open the plugin, hit export and you're done!

Right now I have disabled (commented out) exporting camera animations because they don't really fit anymore, but if there's a need for them, I'll put them back in).

And finally, here's the new interface:

Give a try. You can find the python script in Darkwar's repository: http://svn.icculus.org/darkwar/trunk/base/scripts/

Mirrored for now at: http://home.in.tum.de/~kirschan/blender2md5.py (out-dated but I'm leaving it in case icculus.org is down again and someone badly needs a mostly working exporter)

Either copy it into your Blender plugin directory or manually open it in Blender's text editor and run it with Alt/Meta-P.

Enjoy and Merry Christmas!

Alientrap Application Email

Andreas 'BlackHC' Kirsch — Fri, 07 Dec 2007 20:06:48 +0000

Funny, I've just found the original email (at least the text) that I've sent to Lee Vermeulen when I applied at Alientrap after I saw a screenshot of Nexdm something on flipcode - RIP :-(

Note that this dates back to: Saturday, 23. August 2003, 19:25:14

I'll just tell the truth :

I'm almost 16 years old and live in Germany, but I do understand matrices, etc. I have 8 years of programming experience but only 2 in c/c++. I just learnt OpenGL, so don't expect me to be a John Carmack.

Now let's get to the positive aspects. I want to learn, learn, learn - so give me a problem and I'll work until I've got a solution. Math is not a problem to me, I like it pretty much (especially when I finished calculating some formula).

I've got a adsl flatrate, so I can always search the internet for additional informations.

Perhaps I could be tester, or help finding bugs, etc., until I learnt more opengl and other programming technics.

Sincerely yours, Andreas Kirsch

Also observe the cute mistakes - which I'm still making today, too >_>

But anyway, there were forty applicants and only three 'who survived the "look at the code and try to implement one of the features on the TODO list" test, out of 40+ applicants?' (LordHavoc in IRC). Two of them got jobs in the game industry shortly afterwards, so I was the only one left out of 40 applicants!

Well guys (the 37 of them), you clearly suck, because you didn't even try. I have tried and although LordHavoc pretty much dismissed all of my changes as useless or not committable in the beginning, I still got onto the team and I honestly have never regretted it.

So always: try your best, even if it looks as if you're failing, because you're not the only one and your attempt already is a success compared to simply giving-up up-front!

Now enough of that, enjoy your weekend!

Greetings,
Black

PS: Always a nice read - http://www.thejonjones.com/2005/08/24/smart-people-are-dumb-failure-is-awesome/

xkcd \o/

Andreas 'BlackHC' Kirsch — Fri, 07 Dec 2007 11:48:42 +0000

Today I was waiting for the care-taker to arrive and take a look at some installation issues and while I was waiting I decided to browse a bit around in xkcd. And well... there are a few links that might worth sharing - and I'm not going to link Cat Proximity again this time!

And here are the other ones that I've found interesting while randomly browsing through the archive:

So enough of that, there's more work to be done!

Cheers,
Black

I have a problem that I cannot explain - I have no reason why it should have been so plain...

Andreas 'BlackHC' Kirsch — Thu, 06 Dec 2007 22:32:25 +0000

I have a problem and a solution, but the problem is nice, so lemme tell you about it first.. The problem is as follows:

We have got 31 tracks and 10 trains, Each train is so big, that between two there needs to be at least one free track. Now, how many valid combinations do exist to place the 10 trains on the 31 tracks?

Read on if you think that you have got the solution and want to compare it with mine (no guarantee that I've got it right though..)

My solution:

A valid combination needs to have the remaining 31 - 10 = 21 tracks (after the trains have been placed) distributed between the trains in such a way, that there is at least one (free) track between two trains.

If s_0, ..., s_10 represent the number of free tracks between two trains, respectively for s_0 and s_10 between the outer most trains and the outside world, then s_0, s_10 >= 0 and s_1, ... ,s_9 >= 1 (for a valid combination). Now we can add two tracks left and right that may not be used, so that we can then require s_0, s_10 >= 1, too. That is 21+2=23 tracks distributed on 11 (s_0 to s_10) numbers, and each needs to be >= 1.

Have you heard of ordered number partitions? An ordered number partition is a partition of a (natural) number n into k (natural) numbers, so that the sum of those k numbers is equal to n, and because natural numbers are greater 0, all numbers have to be at least 1.

If you want to determine the count of possible partitions, you can imagine it as sum of n 1s: 1 + 1 + 1 + 1 + ... + 1
That is: n 1s and n-1 '+' signs. We can now group the sum into k sub-sums by selecting k - 1 + signs and separate the sum into sub-sums at that position. E.g.:
1 + 1 + 1 + 1 + 1 + 1 divided in 3 sub-sums: ( 1 + 1 + 1 ) + 1 + ( 1 + 1 )

The number of selecting k - 1 + signs out of n - 1 is simply the binomial coefficient of $\binom {n - 1} {k - 1}$!

So with our numbers we get: = 22 10 = 646646

\o/

Excellent work, soldier!

Cheers, Black

Geckos here and Geckos there - Gecko's everywhere!

Andreas 'BlackHC' Kirsch — Sun, 02 Dec 2007 22:53:53 +0000

Oh a new update and again it's going to be short - probably or at least not proof-read..... Ok, so what about the last screenshot? Well a small update - with moving bits! /

Click it! -- Gecko in Darkplaces with text injection

This is nice, isn't it?

And again this is for EDU2 - an undisclosed, well, maybe it's kinda disclosed now, but you still don't know what it is about, wannabe project after another wannabe project from the Univserity of Jena, which happen to work on with another very talented (it seems!) programmer, who happens to work on Crystal Space, too (hint hint..).

For what it's worth at least I get to spend some time on DarkPlaces again which is a lot of fun because it feels like going to back your roots, since it all started with DarkPlaces more than 4(?!) years ago. I can't believe it's already been that long ago, but that's life.. So even if the project turns out be a failure (chances are 70% if you ask me - you'll get a more elaborate opinion soon), at least and this is the most importan, I work with LordHavoc on something again (semi-again since we only talk about design issues now and then but this is already an improvement). However, what is not an improvement but rather very very awkward, is that I'm wasting so much time with it and actually I'm really horribly neglecting a project (BSS) I've been working on for Joshua, whom I owe more than just a lot. And I feel bad for it. Oh! How bad I feel! Which reminds me to spend the next days on his project and finally get something done again... On other news (or rather similar news) university (the one I actually attend in Munich) is killing my time, too - I have a 30+ hour week at university and add to that the time I have to spend on homework, lecture preparation and wrap-up and I have probably and quite certainly less spare time (it converges to minus zero) than my favourite work-aholics at SplashDamage (yes, you even work on weekends!). Now with the additional projects this actually results in severe sleep deprivation but, well, that's been my choice and in February it'll be over for sure.

So... now I've been talking and writing a lot of thoughts down and I think that's enough for today. More news and screenshots from our multi-touch-screen thingy and BSS(!) soon.

Stay awake and tuned,
Black

Puzzle Piece #1

Andreas 'BlackHC' Kirsch — Wed, 28 Nov 2007 16:15:28 +0000

Ok, lately I didn't have enough time to update this blog and right now I'm still busy with uni and other stuff. What I refer to with "other stuff" is for example:

One of the first screenshots with Gecko support in Darkplaces.

or:

http://vpod.tv/loiclemeur/135867

BTW does anyone want Torque support for Gecko, too? ;-)

Anyway, more updates soon (hopefull).

Cheers,
Black

Oh floating-point comparisons, oh floating-point comparisons - why do you float above causality :(

Andreas 'BlackHC' Kirsch — Fri, 05 Oct 2007 21:46:56 +0000

You know, actually I always wondered why people had problems with understanding why you need epsilon tests for float comparisons: you simply lose precision when you calculate something because floats aren't lossless and so you can't simply use == to compare them, because chances are high that two floats that ought to be equal won't be equal bit-wise. Thus you use epsilon tests instead and only check if both are almost the same (there are many techniques for this though).

I think this concept is easily understandable, but what about comparisons for floats that are bit-wise equal?

Yes, yes, yes! No, no, no!

So let's take a look at this code:

// Update Current Time.
mCurrentTime += elapsedTime;

// Check if the animation has finished.
if (!mConfigDataBlock - > mAnimationCycle && mCurrentTime >= mTotalIntegrationTime) {
    // Animation has finished.
    mAnimationFinished = true;

    // Are we restoring the animation?
    if (mAutoRestoreAnimation) {
        // Yes, so play last animation.
        playAnimation(mLastAnimationName, false);
    } else {
        // No, so fix Animation at end of frames.
        mCurrentTime = mTotalIntegrationTime - (mFrameIntegrationTime * 0.5);
    }
}

// Update Current Mod Time.
mCurrentModTime = mFmod(mCurrentTime, mTotalIntegrationTime);

// Calculate Current Frame.
mCurrentFrameIndex = (S32)(mCurrentModTime / mFrameIntegrationTime);

This is from TGB's t2dAnimationController::updateAnimation function. I had a very weird bug that caused an animation with 5 frames (base 0) and that shouldn't cycle to run through the following frames: start/0 -> 1 -> 2 -> 3 ->0 -> 4/end

Just so you know some initial conditions:

mConfigDataBlock->mAnimationCycle = false, so !mConfigDataBlock->mAnimationCycle = true
mAutoRestoreAnimation = false

So obviously the if-block doesn't get executed for some unknown reason but the mFmod call still wraps the time around and the index jumps to 0 for one frame. Then mCurrentTime increases even more, the if-block gets executed and the time is reset to a value that makes it jump to the last frame (ie. it doesn't wrap now) and then the animations correctly ends.

Now this sounds easy to fix and it is. Simply add an _epsilon test _and you're fine off and everything works correctly. But to make sure that I've actually fixed the bug for the right reason, I did some more test runs with the original code and echoed the value of the floats to determine when the bug occured.

The first little shock was: it echoed: "3.00000 >= 3.00000".

But this is not bad, is it? It's simply the precision error in the float that causes it. The printing algorithm rounds it to a certain decimal for which both values are equal but actually they aren't, right?

Well, so I changed the format string to "%f %f %u %u" and did a pointer conversion to output the integer representation of the two floats, too. They ought to be different now. But the output now was: "3.00000 3.00000 1050253722 1050253722"

Which was very bad for my state of mind. I actually started the debugger and stepped through the function in the disassembler and somehow the greater than or equal comparison would fail to recognize bit-wise equal floats as equal!

The interesting thing was when I added an additional float comparison check a few lines above, the bug wouldn't happen anymore, so I've found a Heisenbug. It gets even nastier: I don't know if you know any assembler, but maybe you have heard of the asm op FINIT, which resets the FPU state and all registers. Now if I add a __asm { FINIT } right before the if-check the bug disappears, too.

So it seems its an FPU bug. Some calculations that TGB performs confuse the FPU and it won't be in a valid state when the float comparison happens.

I've also talked to other coders and general consensus was that you shouldn't really trust the FPU too much and always use epsilon tests (that is some redundancy) just to be on the safe side, because you can't really foresee the state of the FPU at any given time.

TTimo has also mentioned, that the FPU state can be corrupted by other tasks and across task switches, too, so all I'm really certain of now is that this all is some kind of Pandora's box that I don't want to open ever again...

So to sum it up: always use epsilon checks and don't ask too many questions when your code breaks without (if it's possible), because you might not like what you will find out.

Anyway, stay tuned,
Black

PS: I've moved to Munich because of university and I'm only at home on the weekends and only then I have continuous internet access, so updates might become rare again.

GameStar's "Enemy Territory: Quake Wars" review

Andreas 'BlackHC' Kirsch — Wed, 26 Sep 2007 23:10:12 +0000

Ok, today I've got a pretty unusual blog entry. If you're German and interested in computer games, then maybe you have read this ETQW review. If not, well, it's a shame because you have possibly missed one very biased review that really shows gaming journalism at its best. Of course, I'm not and can't be totally unbiased about the game, but see for yourself.

I've taken the freedom to actually translate the whole article into English, so that the non-German speaking folks can enjoy this masterpiece of penmanship, too. It's not so much about the style though (because that surely was mostly lost in my translation), but about obvious mistakes and unfair reviewing... Enjoy!

(Oh before I forget it, this text is my very own translation, so while I've tried my best to preserve the semantics, I might have failed and maybe accidentally removed some errors - also credits and kudos for this goes to our beloved GameStar and its writers... so if you don't like the translation it's all their fault :))

So how many errors did you find?

Cheers,
BlackHC

Enemy Territory: Quake Wars

Splash Damage(sic) mixes the complexity of Battlefield 2 with the pace of Quake - and doesn't satisfy either of the two fan bases with it.

If you crossbreed a horse with a donkey, you get a mule - a quadruped, that has the advantages of both parents, because it is can be used as pack animal as well as a mount. If you crossbreed Battlefield with Quake, you get Enemy Territory: Quake Wars - a multiplayer shooter, that has the advantages of both parents, because.. Wait a second, exactly not! Because Splash Damage's attempt to merge two different multiplayer games with obviously different kinds of gaming experience has only been partially successful. And this disastrous crossbreeding is not the only issue the game is having.

The plot: slim

The developers didn't hold themselves up with an extensive background plot: The alien Strogg attack mankind and spawn conflicts all round the world. Similar to Battlefield each of the twelve maps tells a small bit of the story of the bigger conflict. Hence in "Valley" you have to keep the Strogg from contaminating the drinking water supplies of the American West coast. In "Island" the human forces have to shoot their way through a marvelous beach to the enemy base, steal important data and escape. And "Slipgate" sends you to a small African village, where you have to hack a Strogg slipgate and suddenly arrive at the opposite end of the world (see the box with the "example mission").

In doing so primarily the level design is convincing. The maps are diverse and some are spectacularly built - surprising what the developers could get out of the three years old Doom 3 graphics framework. Even more surprising that of all things all the interiors (you would think one of the engine's strengths) are designed rather dully, textured unsightly and lit weakly.

The heroes: plump

No multiplayer shooter without classes. Quake Wars is no exception here. And has no surprises: The Soldier of the GDF and the Aggressor of the Strogg take heavy equipment into the battle, the Medic/Technician heal wounded teammates, the Engineer/Constructor repair damaged vehicles, the Field Ops/Oppressor deploy turrets[sic!] on the battlefield and the Covert Ops/Infiltrator shoot around with high precision weapons. Sounds like a system designed for teamwork, but it has issues because of design quirks. For instance you normally don't need a medic, because its faster to wait for respawn instead for the rescuing defibrillator.

Because the team isn't punished for the death of a teammate with a deduction of points, there are hardly any heroic medics on the battlefield. The Field Ops of the GDF who helps his teammates with ammunition packs is also only of limited use for teamplay because you die very fast in combat as usual for quake games and thus you'll rarely run out of bullets anyway. While we're talking about dying: We would have appreciated more feedback about the origin of the final shot, too. It happens too seldom that you know why you're suddenly lying on the ground.

The missions: cool

The only sensible use for the classes are the cool missions that you need to complete in the course of a round. For instance only the sniper can hack a shield generator. This is necessary in "Valley" where a mobile command post (MCP) will only fire its missile at the Stroog base when the latter isn't protected by a shield anymore. To get the MCP into position, an engineer should be present to repair the defenseless machine after it has been damaged by enemy fire.

This idea distinguishes Quake Wars from the usual flag capturing of a Battlefield 2 and pleasantly concentrates the action onto the currently important area of the map. In the process the developers have now and then overshot the mark. If for example the Strogg try to defend the branched tube system in "Sewer", 32 players race around shooting at each other - not a bit of tactis or teamwork in there.

The weapons: not cool

Quake Wars has everything from a machine rifle over a shotgun and a nailgun to the legendary railgun that should keep Quake fans happy. But the awesome weapon balancing of the original is painfully missing though. If you are playing as a Strogg for example and try to shoot through your scope, the zoom is pixelating everything for a moment before the view becomes clear again. In a fast paced game like Quake Wars this often already causes death. Generally all Strogg weapons seem weaker than their GDF counterparts. The railgun of the Infiltrator for example has a stronger recoil than the sniper rifle of the Covert Ops.

A hint of tactics is creates by the extras: Especially the freely placeable and rotatable turrets are fun, particularly since some of them trigger super weapons like the ion laser[sic]. On the other hand lots of it is really uselss: The stationary camera for example was never needed because it doesn't have any advantages except for the small reconnaissance area and it takes too long to deploy. The mines are likewise only of limited use because you have to place them first and then activate them. In a game like Battlefield that tries to be realistic this would be sensible but in Quake Wars it doesn't suit the high speed of the game.

The vehicles: fast

On most of the maps a variety of land and air vehicles is available - from the unarmed but incredibly fast Quad to the Cyclops, a heavy armored Strogg mech. From outside all the vehicles look worse than in Battlefield 2. When you enter them though, you get to see the nicest cockpits of the genre. The driving experience is ok, too. The only exception is the tank of the GDF, whose mass you don't notice because of the mediocre game physics and the fact that is tossed around like a rubber ball, when it explodes.

Apropos explosions: Especially on maps with lots of vehicles an exceellent combat experience is created. Debris is falling from sky, vehicles are exploding and it's cracking and Banging everywhere. A bit more work on the effects would have been nice though - the explosions look flat and hardly spectacular.

Score:

General Conclusion: Although we miss the teamplay experience, the online battles are still fun.

Rating:

Graphics:
+ sharp textures
+ incredible view distance
+ good animations
- partly sparsely detailed landscape
- moderate effects
- frame cap
8/10

Sound:
+ cracking explosions
+ good English voice actors
- bad German voice actors
- partly dull weapon sounds
8/10

Balance
+ maps are mostly balanced well
- unfair bottlenecks
- Strogg are a bit weaker than GDF
6/10

Atmosphere
+ nice battle atmosphere
+ gloomy end time scenario
- teamwork experience is hardly building up
- radio messages get annoying after some time
7/10

User Interface
+ intuitive shooter controls
+ well arranged server browser
+ useful command map
- fiddly quick chat
9/10

Game Content
+ 12 partly big maps
+ 3 game modes
+ 5 classes
+ numerous gameplay options
10/10

Level Design
+ very diverse
+ infantry and vehicle maps
+ pretty realistic settings
- dull interiors
9/10

Teamwork
+ different missions for different classes
- help is not rewarded
- Medic/Technician and Field Ops/Oppressor are only of limited use
5/10

Weapons & Extras
+ big weapon selection for every need
+ cool turrets
+ popular Quake weapons
- some extras are useless
9/10

Multiplayer Modes
+ more thrilling than usual flag capturing
+ difficult tasks
- only stopwatch mode interesting for online matches

Conclusion: Action-filled online shooting without teamplay experience (79%)

Game in a Day Event September '07

Andreas 'BlackHC' Kirsch — Mon, 17 Sep 2007 23:15:57 +0000

The Game

Oh dear, it's really been a while since I've found time and will to write anything for this blog. But things change and I'd like to write down a few things about this month's GID event.

First, here's a cross-link to the GID's board: http://www.gameinaday.net/viewtopic.php?t=122

Now the game is called Fireworks! and it's about, well you might have guessed it, fireworks, but not really. I've rather taken a more unrealistic approach and thought about what fireworks look like in cartoons, etc. There they often explode to some drawing or shape, which of course is pretty much impossible. But the idea is nice, and since I've wanted to do some mouse-based/movement-based game, I thought that doing a firework-themed drawing game would be fun.

See the screenshot on the right for what the firewor-theme looks like. You first draw over a shape that is shown and when you are finished the firework starts. You do this to collect as many points as possible to get into the highscore list (doesnt sound like duh?).

The actual idea consisted of actually drawing stuff freely and mistakes would appear in the final firework, too, but I had to give up that idea because I couldn't get T2D to render particles at a certain quantity at low FPS, too, and I didn't want to use separate objects for it. Maybe the invalid strokes will come back , because I kind of like the idea of making invalid strokes more visible.

The score system is pretty ok but the UI suffered because of the time constraint and isn't as dynamic as it should be, but you can get boni and multipliers for lots of things.

Original brush-based approach

Enough said, here's the link to the current final build (there might follow one with sounds later): http://data.blog.blackhc.net/gid/0709/Fireworks!0.99.zip

And here's an excerpt from the readme:

Preliminary Stuff

This is the almost or probably final version of Fireworks!, a GID game (15/16.9.2007).

Pure dev time: 19 hours atm spanning from Saturday till Monday.

The game is released as is, etc.

Most of the source scripts are included, so you can take a look at a few bits pretty clean code midst very hackish stuff.

About the Game

It's pretty simple: You have to draw over the contours visible in each level to earn points and start/arrange the fireworks (that's pretty much the story).

The score system is pretty simple: The longer you stay on a contour (the cursor trail is cyan), the more points you get, and after many points in a row, you receive a power multiplier as reward. Likewise, if you draw fast enough over a part of a contour that hasnt been painted yet (purple at the beginning, green if you've already drawn over it), you'll receive a speed multipier. Also if you manage to finish multiple contours with one, you'll get a combo multiplier.

Each multiplier starts with x2 and increases if you draw accordingly.

But be careful! The multipliers are only applied to the score of your stroke, when you release the mouse button! They add up and multiply your score: e.g. +1000 x + 2 + 2 means 1000 * (2+2). And if you miss a line or your mouse slips a bit and the cursor trail turns red, you'll lose the combo and the power multiplier (not the speed one, though)! So be careful and don't try to make too big a combo or draw too difficult a contour at once because you could lose lots of points...

If you manage to paint a whole contour line with one stroke, it'll double your current score and also increment your combo multiplier. Same if you manage to paint all contour lines with only one stroke per line. Last but not least, one last word: there's a highscore and there are 5 levels at the moment. There is no game-over condition yet, so you can simply play through all levels and try to get a good highscore.

Post-mortem

Don't always get lost searching for some stupid tools that aren't really that necessary for the project

Ok, this time I can't say that I did a lot of bad things, although I think I've spent way too much time trying to find a good GIF animator to build the gif movies with (quite possibly 4-5 hours over the course of the last three days).

Beware possible feature creep and rather try to cut and stay focused on the game vision

Development itself went fairly smoothly although I almost got lost with the particle brush feature: I probably spent 3 hours on it, until I realized that it wouldn't work under the setting I've been planning for it, and I had to refocus on the actual game instead of toying around with it some more.

Limit yourself on the engine and try to stay inside the game script side of things

I think it was a good decision to use vanilla TGB 1.1.3 (although I maybe should have tried working with 1.1.5 for this game) and not doing any engine changes, because it helped me concentrate on the important gameplay ideas instead of implementing everything cleanly and properly. Plus it kept me from debugging the engine at all. On the downside it again took me half an hour to remember the parameter passing bug in eval statements in vanilla TorqueScript and use the work-around in my code. Also the t2dPath implementation is horrible and buggy! And I thank god, that it didn't blow up all the time and I was able to correct the mess ups with a text editor or simply recreate the level.

Be flexible about the game design and simply keep it simple, stupid (again!)

The original game I envisioned isn't all that different from the outcome but it still is different enough that I wouldn't have imagined it to be this way. The design changed a fair bit with me making up my mind on it only Sunday morning (while showering). I spent most of Saturday night with feature proof-of-contest coding and setting up everything for the actual gameplay code. Being open to changes helped me a great bit, because I would simply adapt the game look and design to limitations (both time and feature), and keep on creating a playable game.

Try to reward the player with nice graphics and sounds

I think the visuals help the game a lot because you really rather play something that looks nice. At the same time it's a pity and a real problem that the game doesn't make use of any sounds or background music and this hurts it quite a lot.

Good/intelligent level design is even more important for small prototypes

Another thing I only learnt a short while ago about is that the order of the levels (something I spent less than a minute on) had a huge implication on how the gameplay was perceived by players. Because the the first levels can be solved by making one continuous stroke, many players thought that this was the main task of the game and were confused when they played on. For me the level design simply was something to do half an hour before the planned release because I was only concerned with the code and showing that it works well, whereas actually the level design is more important than the code because only it determines how the game is viewed (fun or boring). I certainly have to spend more time and consideration on level design at the next GID.

All in all though it went fairly well and even if the game might not be as much fun as it could have been, I still am of the opinion that it looks and play fairly professional for a game that has been produced in less than 20 manhours.

That's it for today.
Over and out,
BlackHC

Coroutines

Andreas 'BlackHC' Kirsch — Tue, 17 Jul 2007 22:01:35 +0000

Have you ever heard of Coroutines? I haven't until a few months ago when digibob (from SplashDamage) mentioned them and explained to me what they are. Instead of pasting his explanation from my IRC logs, I'll instead quote Wikipedia:

In computer science, coroutines are program components that generalize subroutines to allow multiple entry points and suspending and resuming of execution at certain locations.

[...]

Coroutines are more generic than subroutines. The start of a subroutine is the only point of entry; the start of a coroutine is the first point of entry, and places within a coroutine following returns (yields) are subsequent points of entry. Subroutines can return only once; in contrast, coroutines can return (yield) several times. The lifespan of subroutines is dictated by last in, first out (the last subroutine called is the first to return); in contrast, the lifespan of coroutines is dictated entirely by their use and need.

(from http://en.wikipedia.org/wiki/Coroutine)

Let's continue with an (pseudo-code) example:

coroutine unsigned coFibbonacci() {
    unsigned a, b;
    a = 0;
    b = 1;
    while( 1 ) {
        unsigned c;

        YIELD( b );
        c = a;
        a = b;
        b += c;
    }
}

Calling coFibbonacci repeatedly would return the Fibonacci series. A more advanced example (which I won't paste here) would possibly consist of game scripts that control scripted sequences and always yield when waiting for operations to be performed, so that the control flow will look natural while basically being event-based. Usually one uses state machines in languages that don't support coroutines but an implementation using coroutines would be a lot more elegant.

Many scripting languages have coroutine support, like e.g. Python or Lua, while coroutines are widely unsupported in most other "normal" languages (notable exception .NET). I have read some of the Python coroutine/generator specs in more detail to figure out some implementation details and the example code is also very neat to see the advantages of coroutines when traversing trees (for example) . In particular I've read PEP 255 - Simple Generators and PEP 342 - Coroutines via Enhanced Generators.

Seeing all the advantages I wondered whether and how difficult it would be to implement basic coroutine support in C/C++. The answer depends on several factors. First of all it is possible to write an implementation that doesn't use evil setjmps or asm and that makes use of some C/C++ features in clever ways. My pure C/C++ implementation makes use of Duff's Device to implement multiple entry points and static variables to save the "locals" between calls.

It's pretty small and simple:

#define YIELD( value )
   do {
      yieldIndex = __COUNTER__;
      return (value);
   case __COUNTER__ - 1:
      ;
   } while( 0 )

#define RETURN( value )
   do {
      yieldIndex = -1;
      return (value);
   } while( 0 )

#define COROUTINE( returnType, name )
   returnType name() {
       static int yieldIndex = -1;
       switch( yieldIndex ) {
          case -1:

#define ENDCOROUTINE
         yieldIndex = -1;
      }
   }

PS: it should actually work with __LINE__ instead of __COUNTER__ and __COUNTER__ - 1 but VC somehow doesn't like it
PSS: the backslashes in the macro definitions were removed by Wordpress

Recently my interest for coroutines has been renewed and I created a macro-based implementation that would allow multiple "generators" to be created and run in parallel. For this the stackframe of the coroutine had to be saved and restored later and normal locals instead of statics had to be used.

The main issue with this implementation is that Duff's Device couldn't be used anymore since VC will correctly warn (and error out) that it is told to jump over local variable declarations - which of course would be very bad normally.

Thus I had to write custom __asm inline code to perform the jumps. Moreover when yielding the code has to jump over local deconstructors to avoid early clean-ups of the stackframe.

This version is a lot bigger and because of the use of the __asm keyword not very portable (although the asm code itself is very primitive and should run on any processor):

#define COROUTINE( returnType, name, paramList ) class name {
   size_t yieldOffset;
   void *stackCopy;
   size_t stackSize;

public:
   name() : stackCopy( 0 ), stackSize( 0 ), yieldOffset( 0 ) {
   }

   ~name() {
      /* this is something bad actually - it should always finish but oh well */
      if( stackCopy ) {
         delete stackCopy;
      }
   }

   returnType operator() paramList {
      size_t __yieldExitOffset;
      __asm { mov __yieldExitOffset, offset yieldExit_##name }
      returnType __yieldReturnValue;
      char __frameStart;

      if( stackCopy ) {
         /* restore the stack */
         memcpy( &__frameStart - stackSize, stackCopy, stackSize );
         size_t localYieldOffset = yieldOffset;
         __asm { jmp localYieldOffset }
      }

#define CONCAT( a, b ) a ## b
#define LABEL( line ) CONCAT( __label__, line )

#define YIELD( value ) do {
         char __frameEnd;
         /* copy the stack frame */
         if( stackCopy ) {
            delete stackCopy;
         }
         stackSize = &__frameStart - &__frameEnd;
         stackCopy = new char[ stackSize ];
         memcpy( stackCopy, &__frameEnd, stackSize );

         __yieldReturnValue = (value);
         size_t localYieldOffset;
         __asm { mov localYieldOffset, offset LABEL( __LINE__ ) }
         yieldOffset = localYieldOffset;
         __asm { jmp __yieldExitOffset }
LABEL( __LINE__ ):
         ;
         } while( 0 )

#define RETURN( value )
   do {
      /* remove the stack copy and clean up */
      if( stackCopy ) {
         delete stackCopy;
         stackCopy = 0;
         stackSize = 0;
      }

      yieldOffset = 0;
      return (value);
   } while( 0 )

#define ENDCOROUTINE( name )
yieldExit_##name:
      return __yieldReturnValue;
   } /* function done */
}; /* class done */

PSS: again backslashes were removed by Wordpress

If you're interested in some example code, here's a zip with my VC 7.1 project.

The only known and unfixable bug in the latest implementation is that it doesn't adapt pointers to local variables between yields (but for this real compiler-support would be needed).

Greetings,
Black

Game in a Day Event (June '07)

Andreas 'BlackHC' Kirsch — Mon, 18 Jun 2007 22:02:33 +0000

This weekend I've participated for the first time in the GID event. The common task is to create a running game prototype in 24 hours (24 hours pure developement time + naps is allowed, too).

I think it went well considering that this was my first participation but essentially I've failed because my game prototype is nowhere near finished. It's rather at an early proof-of-concept stage instead...

Let me introduce the prototype/game quickly - it has no name yet, so let us look at some screenshots instead:

Teaser

A running stickman

You see, it's about a stickman that can move. I'm not going to go into more details yet because the game doesn't consist of much more right now. If you want to see a more interesting animation though, click here.

For those interested here's a postmortem.

Postmortem

The first problem arose right when I wanted to start: what should I code?

I've come up with a few different ideas the days before the GID started: a Zone66/Baryon remake (space-shooter like), a Prince of Persia remake and/or a platform game in general or a game which was consisted of drawing shapes with the mouse - I've played a few very neat games of this kind lately: Chalk and Crayon Physics - and I felt that this was an idea that I'd really love to explore but on the other hand I couldn't come up with an interesting and original gameplay idea.

Beforehand I've prepared a few things for a possible platform game. I really love stickman flash animations (for example the one here) and thus I've decided that a stickman would be the perfect protagonist, especially since it certainly is easy to create good-looking animations for him.

Searching around with Google, I've also found a very neat tool called Pivot Stickfigure Animator, which allows one to easily create animations for stickfigures and other similar shapes and export them to animated GIFs.

An idea I had Saturday morning was a game in which the stickman is running all the time and you have to do cool moves to avoid obstacles and maybe fight enemies. This should create tension and force the player to always pay attention while rewarding him with neat animations and fun gameplay. The feelings created in the players should be similar to those I felt when playing Prince of Persia: Warrior Within when you're trying to escape from the time guardian - I really love these parts of the game! - but at first I thought that the idea was not worth pursuing. Later though, Juz from #q3mods proposed the same gameplay idea and I told myself that this can't be a coincidence and decided to create my prototype based on it.

So I've only been able to come up with a good game idea late into Saturday afternoon while the others were already working on their games which is the first thing that went wrong: unsufficient preparation.

The next one was that I concentrated too much on creating art and sadly the results aren't stunning despite the time I've spent on creating the animations. A reason for this could have been my horrible and most terrible art pipeline which looks like this: Pivot Animator -> GIF Constructor -> GIMP -> GIF Constructor -> GIMP -> TGB This couldn't be automated and every step had to be done by hand.

After about 10 hours of work the game was this far. I've also lost quite some time (~2.5h) developing a movement cache system that wasn't used at all because the game never got far enough to support more complex moves and combos which is another thing that went wrong: wasting time on useless stuff.

It also turned out that although I have already developed a few games in TGB I still didn't know some systems well enough (mainly collisions and physics) because I've never coded a platform game before and I had to use trial&error quite frequently while implementing new features. Which leads to another thing that I could have done better: know your tools beforehand!

And last but not least, now that I'm looking at my end-product, I realize that I haven't stayed with the simplistic gameplay idea I've had in the beginning but instead diverged to a too complex game ; thus last two items on my list of stuff that went wrong are: violation of the KISS (keep it simple, stupid!) principle and feature-creep.

On the other hand there are a few things that went right, too: This is the first time I've developed a game using rapid iterative development and it worked surprisingly well: hack a few features into the game first as fast as possible, then clean-up and refactor the code afterwards and repeat. (Normally I'm attempting to write the clean version directly but this takes more planning.)

Moreover I've learnt many things about TGB I haven't known before and I've grown even more fond of it as a game development system. And ultimately I'm going to continue development of this game even after the GID's deadline because I really like my stickfigures.

On other news I've finally finished my A-Level exams and it seems that I've really rocked them. Tomorrow I'll be allowed to take a look at the marking and hopefully that will explain why I got such good marks..

About Code, Quality and General Hackability

Andreas 'BlackHC' Kirsch — Mon, 07 May 2007 21:08:44 +0000

Good code is elegant.
Good code is short and to-the-point and offers the simplest solution to a certain problem.
Good code is easy to maintain and to understand.

What is good code??

I've always been trying to write the best possible code and I think in the past years I've learnt quite a lot on how to write code that in my opinion is good code - mainly due to guidance from great books like Code Complete and Programming Pearls and from reading other's code (mostly id's) to see how they write their code and from reading other's to see how not to write code because it was hard to understand or a plain and utter ugly hack.

There always comes a moment for every programmer now and then to go out of your regular thinking patterns and reflect about all the code that you have written and really evaluate its quality, its advantages and its disadvantages.

The moment seems to be here right now for me.

In general I really hope and believe that my coding style has evolved to the better in the last years but there is certainly a lot that I need to criticize if I'm honest.

The first and biggest point is that the code I'm the most proud of seems barely readable and baffling to most of the coders I've talked to (- all of them to be honest).

Why?

The code I'm the most proud of is elegant, works and I can extend it easily and after reading a few bits I quickly understand its design again. It also is very complex but it arguably does an at least moderate if not good job at dividing the problems into different subproblems, defining interfaces, separating everything from each other and so on (see Code Complete for a list of things it tries to achieve and possibly achieves). The one thing it probably and quite certainly lacks is good documentation. I'm not a fan of writing lots of documentation because code changes and updating comments is a tedious process you're often forget to do. This naturally leaves the readers of your code more confused than without misleading comments in the first place.

I've been trying to make my code document itself - look at DarkWar's shell code to see what I mean: the function names are more than descriptive, the code reads like a book and it is its own flow diagram pretty much. Still people are frightened by it and most of Nexuiz's coders are scared by the menu system I've written.

The problem as I see it currently is, that although I haven't over-engineered the code, I certainly have taken too many measures to make it only work as intended. That is, the code will fail when you try to hack it - which in my opinion still should be a good thing, but apparently isn't.

More precisely, the design doesn't allow one to add hacks, because it's so tight and so self-contained in its thought-model that there is only one way to make changes: make them correctly and have them fit the abstract design concept or if that doesn't work, change and adapt the whole model to your changes.

I consider this great, since, if you know the design and the code, it prevents you from doing stupid things and provides an easy way to add features, since there are only correct ways to do it - albeit only few though.

The problem with this is, however, that you need to allow other people to hack your code and try things, as this is the way most people seem to learn and understand new designs: (intelligent) Trial & Error until they find the right solution.

If you don't, people will be scared by your code, because from their perspective it is like a brick wall that doesn't allow any changes to its structure.

There are two solutions for this issue:

Tell people to read your code like a book first and then start to carefully change it, but this is what I've been doing most of the time and it certainly doesn't seem to work
Try to change your code to fail more gracefully and allow more hacks: soften pre- and post-conditions in functions, maybe weaken the design contracts, and so on. All in all: allow more holes in the code design.

I don't like either solution to be honest, but I'm still searching for a satisfying answer.

I only know that writing code that allows hacks too easily is bug-prone and quite probably a hack-magnet and will lead to equal confusion among readers because of the lack of a strict (abstract) design (as was the case for me in some parts of DarkPlaces) and it makes coding-life - once you know the codebase - more difficult too, because you'll have to fix more bugs. Writing non-perfect code can turn out fine, too, though, as I've experienced during Sploidz's development - some of the designs and parts of the code are a bit hacky but all in all it still leaves a good feeling when I'm reading it and it has allowed for fast development - and Shelled's development - server-side mostly though since the client-side really needs a rewrite -, but one has to carefully measure every hack and loophole one allows.

To sum it up before I write another ten pages, code has to be hackable and it shouldn't provide too much resistance against bad changes and hacks at first to let people toy around with ideas and get comfortable with the code but then it should still limit them to writing good code in general by making the hacks fail in the big picture, i.e. for small hacks it should work but if you want to make it work for real, you should need to do it right and the code should also point you towards it, that is, the code layout should offer a guidance for the changes you want to make.

There is already lots of code out there that does this and also lots of code that is going to be released, so I hope that I'll find some more insight on this soon :)

All this rant has been created from a simple insight while I was taking a break from learning for my A-level exams, and I hope that I'll be able to investigate this some more when I'm done with my exams.

Anyway, stay tuned,
Black

PS: My first exam is in English, so this hopefully ought to be some good and valuable practice.

Oh school oh presentations!

Andreas 'BlackHC' Kirsch — Tue, 27 Mar 2007 16:29:22 +0000

Again I've been busy with school. I had to prepare a presentation about Thomas Bernhard for German class and I've spent the last 4 days in my room reading his books, that is "Gehen" and "Der Untergeher", and reading his biography and reading more of both and creating a presentation that contained 60(!) slides in the end. Presenting took me awfully long - more than 70 minutes - but at least the teacher thought it was a fine. You can find an example of Thomas Bernhard here (although the translation may be not that good).

I've uploaded the source and the presentation to my webspace.It released under FDL. You can find it here. Doing this I've come to think that my other physics presentations might be worth uploading, too, so here they are:

one about the Doppler effect: here
one about optical resolutions, telescopes and microscopes (that makes use of some nice java applets): here

They are all written in German. Anyway, I hope someone can use it or read it or like it. Fortunately I'm going to work on some computer-related topics again now - although my A-levels exams will begin soon :(, but until then I'll try to code.

Greetings,
Andreas

PS: I've also uploaded my entry for the BWM (Bundeswettbewerb Mathematik - this is a national math contest) in 2007. You can find it here.

Shelled Project Pages

Andreas 'BlackHC' Kirsch — Thu, 08 Mar 2007 12:33:03 +0000

I've finally found enough time to write a few things about Shelled and put up a general TODO Shelled info page and one describing my memories of its development. So if you're interested you can take a look at it and tell me how you like them respectively how you like the game. By the way its totally free and you can download the game source and mod it, too, so if you haven't tried it out, do now!

On other news I've been working on this and that and my GarageGames project still isn't finished as I'm waiting for some artwork and I've been playing around with Doom 3 and Quake 4 (respectively their SDKs), maybe I'll post a few screenshots in a few weeks. I'm still in a trial-and-error phase but I know what I want to do, so hopefully I'll be able to create something nice. There are also a few other projects and tasks that keep me busy: mostly school at the moment and preparing for my A-levels - though I've only started last week with that -, the second round of a computer science competition and other stuff.

Anyway stay tuned for more info soon.

Black

Senior papers and what's all the fuss about them anyway..

Andreas 'BlackHC' Kirsch — Mon, 08 Jan 2007 14:20:13 +0000

Yeah, finally I've finished my senior paper and handed it in today. So what's all the fuss about senior papers anyway?

Here in Germany you have to write a senior paper in the last grade before A-levels in one of your two main courses. It has to be similar to a scientific paper about a topic you and your teacher choose. Typically it's about 10 pages long but most often it seems to take about 20 pages.

It's probably the longest non-coding project I've ever done and I really hope it'll stay this way for some time since I'm not thrilled by the thought of doing anything like it again (at least for the next 3 to 5 years). The same applies to all my classmates, although its probably the longest project they've ever done for them. Everyone has been writing and printing and correcting and printing again constantly for the last 2 to 3 weeks and this has literally been crunch mode for most of us.

I'm actually even feeling a bit burned-out because of it since I've been spending most of my vacation working on it instead of relaxing and now after one day of school I again feel ready for some real vacation..

Anyway, I've uploaded my senior paper and you can find it here. However, currently there exists only a German version of it, so if you don't understand German it will be of little use for you. I'll try to create an English version of it with LaTeX when I find some free time but I can't promise anything.

The sources to it are released under the "GNU Free Documentation License" and can be downloaded here.

The senior paper is about "creating graphical simulations with a pc" and describes some of the techniques required to create a raytracer. (The German title is "Modellierung graphischer Simulationen am PC".)

Stay tuned for some more coding-related entries soon but I'm still busy with another school project so it make take a while before I'm back to normal.

Black

XML, XSLT, HTML and what it has to do with Torque GameBuilder

Andreas 'BlackHC' Kirsch — Tue, 05 Dec 2006 14:52:25 +0000

Good title, eh?
But really, it seems like a lot of interesting buzz words were thrown together to create a leet-looking tagline and yes, that is my original intention.

On the other hand I really had to do with all those buzz words, simply because I was too lazy to start up Doxygen... You see, Torque GameBuilder(or short TGB) didn't use to have very good documentation or rather the function and class reference was always at least a little bit obsolete, so when I needed a good and fast-loading reference, I preferred my self-created .chm file over GarageGames' pdf reference - which was also awkward to navigate in. TGB has console functions that output all console functions and classes in doxygen-friendly C++-parser compatible data and you can feed that to doxygen and create a nice reference from it, so that is pretty simple job and gives you a very useful custom reference nevertheless.

But as it comes I'm now using Torsion (a nice little IDE for TorqueScript) and it has its own way of extracting the symbol information from TGB and storing it in an .exports file, which is nothing else but a renamed .xml file. XML stands for Extended Mark-up Language and is the generic big brother of HTML, if you haven't heard of it.

Knowing this and being in dire need of a new reference file, I thought "Heck, I'm too lazy to take the 10 minutes and 5 relatively-easy steps to create my usual doxygen documentation for the new release - no, I'm somehow going to use the XML data to create my documentation! And although I don't know anything about XML in general or any tools for working with it, it certainly will be a lot more entertaining, time-saving and fun than the doxygen way!".

Well, sometimes I'm as dumb as you can get - but not this time! Yes, it drained all my resources for half a day and yes, eventually I started up doxygen and did it the usual way, but... - ok, lets rephrase that: Although I haven't been able to use it productively after all, I've learnt a lot and it certainly will help me some other time in my life. *puh*

As I'm owning Visual Studio .NET, I was convinced that it would offer some way to edit XML data and represent it in a human-readable way. I've used Torsion's exports xml from a free TGB game (you can download it from www.garagegames.com) called Marble Tactices and here is a little excerpt:

<class>
<name>ActionMap</name>
<base>SimObject</base>
<method>
<name>bind</name>
<args> device, action, [modifier spec, mod...], command </args>
</method>

This isn't exactly readable and after opening it in VS.NET I spend a long time trying to figure out how to control its representation. After approximately half an hour I stumbled over the XML stylesheet language or more correctly: XML stylesheet language transformations (XSLT).

Here's the same excerpt from above after it has been transformed by one of my XSLT files (or rather by msxsl with one of my XSLT files specified as stylesheet):

ActionMap SimObject

Methods

bind( device, action, [modifier spec, mod...], command )

XSLT is an XML-based format that can be used to describe the conversion from one XML dataset to another. HTML or rather so-called well-formed HTML(HTML that adheres to the XML standard) can thus be seen as the output dataset and consequently you can use XSLT to describe the conversion process that transforms the content of a certain type of XML files into concise and human-readable data (human-readable as in "pleasant to the reader's eye").

Microsoft's help is very neat and I only took me half an hour to have the first basic version working and it probably took longer to learn HTML and CSS (knowledge which I haven't actually used for my final PoC) than to understand XSLT as concept thanks to very good examples and good documentation. Talking about documentation, if you need to learn HTML, XML, CSS or any other web-related standard, W3C is the best place to find specifications and help.

Well, maybe it's just me but it's a lot easier to learn something new comprehensively if you read the specs rather than some whacky tutorials for beginners that you can find on the net everywhere...

Anyway, XSLT is great, great and again great like a lot of things that have to do with XML since they are very easy to use. XML really seems like a good way to go, since there are many nice generic tools available for it and one doesn't have to reinvent the wheel all the time.

If you find time to look at my XSLT "code" and the output it's capable to create, you'll realize what a time-safer it could become, if XML becomes more widely used (in games programming).

Writing all 3 XSLT PoCs/tests took me a bit over half a day and finding, learning and getting mktree to work properly took one third of that time, so you see that XSLT really has got a nice learning-curve. For those who don't know mktree, it is a free javascript library that you can use the create tree structures in your HTML/DHTML files.

You can find and download my Torsion exports -> HTML XSLT example (3 proofs of concept) here. BTW Everything else of importance is explained in the readme.txt.

Enjoy and stay tuned,
Andreas Kirsch

Wohaa!

Andreas 'BlackHC' Kirsch — Fri, 17 Nov 2006 18:41:49 +0000

A lot has happened in the past weeks and I still can't believe it. But first things first:

For the second time I'm taking part in Germany's federal computer science contest (for students): It's called "Bundeswettbewerb Informatik" and consists of 5 problems (respectively 6 for teens aged under 16) which have to be solved in about two months' time. I wrote and tested everything in about 4 days (2 days in my fall vacation and 2 days right before the submission date). The code isn't beautiful:

it's meant to be short, so it won't waste too many pages when printed out;
it's meant to be simply and make use of libraries, so the documentation can be kept short and to the point (no need to explain sorting when STL already does it);
it was hacked together in only 4 days in semi-crunch mode.

The documentation isn't very good either, for the aforementioned reasons and I wrote all ~14 pages in 2 days, so there wasn't much time to correct and proofread, but anyway you can download my submission here.

(BTW if you happen to be a corrector for BWINF and specifically happen to be mine, please note that I always exaggerate and am pessimistic, and my submission is pretty good after and I've taken it very seriously *nervous laughter*)

If it wasn't for my job application at Splash Damage, of which I unexpectedly heard back four days before the final submission date of the contest, I would quite possibly not have taken the time to finish 4 problems in total, write some lousy documentation for them and send them off, but the motivation boost that the reply from SD gave me (or rather the prospect of it) was simply incredible. Now the answer to my application is a whole different story and I'll tell it some other time.

In other news, Shelled! has finally been released Tuesday night and I've been playing it like mad online and people seem to enjoy it, too. I guess we can call it an success, even though there are still some minor bugs in the release build. But Joshua, the owner, founder - and the honorable master of Red Thumb Games found a job/internship at GarageGames and has been busy since, so he decided to release the current version of Shelled! for free instead of waiting another month or two until some kind of copy protection had been built in by us.

Download it from http://www.shelledgame.com and give it a try!

I'll try to write a post-mortem soon and add it to my project page, but right now I'm busy with some contract work I got from GarageGames (thanks Joshua!) and I have already spent enough valuable time writing up this post :)

So stay tuned and watch out for the fish,
Andreas

Tired Fish

Andreas 'BlackHC' Kirsch — Mon, 30 Oct 2006 00:18:47 +0000

I have really, really never imagined that my life would become that boring coding-wise after school starting again. There is almost nothing to tell since the last post. I've worked a bit more on the raytracer but eventually decided that I should sacrifice completeness over finishing my senior paper I was writing it for, since everything I wanted to put into the paper was already in the codebase. So I finished it: right now it's 10/11 pages long and contains a lot of matrix math, etc. and no direct code at all - even the description of the inner-workings of a raytracer is held very generic, so that hopefully even my math teacher, who doesn't know programming languages very well (- she probably only knows Basic or Pascal), will understand it.

You can find the current version of the senior paper here, but be warned, it is written in German, so you might only be interested in it, if you know German or like looking at cool formulas and proofs.

I won't upload the raytracer right now, since it isn't finished yet.

What else can I tell you? Well, if my coding life hasn't been very productive - to put it more accurately my fish almost died of boredom - my other life (you could call it "social life", but I'm not sure about how correct that would be..) has been exceptionally busy: I broke my other wrist, too, and had to run around with a cast for the last three weeks. My birthday was also in the beginning of October and my birthday party, too :)

The aftermath of it is still frightening me, but first things first. Since a good friend of mine has the same birthdate as I, we decided to celebrate together. We rented a few room, invited about 70 people and hoped for the best. Thank god, only about 40 came, but the party was still pretty wild and many bottles of beers broke..

The rooms were rented from the Red Cross, but we didn't know that the actual owner of the house was the next-door neighbour. He apparently came over two times during the night to tell people that were outside to be quieter, but he didn't talk to us, and only on the third time he asked for us and threatened to call the police if we don't leave his house, yadda yadda. A nice ending for an otherwise good party.

It came worse: The guy from the Red Cross was woken up by the owner early morning and was angry, too. Normally we would have had another 10 hours to clean the rooms and give everything a polished look again, but he retured way too early, looked at everything and gave me a pretty bad time. Not that I cared much, I didn't drink a lot that night: being in the kitchen preparing pizzas, mopping away spilled beer and taking responsibility usually leaves you sober. Additionally I didn't sleep one minute and being in such a zen-like state of mind, the guy could have mutated into a twenty-headed hydra and I wouldn't have been surprised. If he had exploded into pieces, I may have applauded, but of course only because of my dark sense of humour.

Needless to say that we lost the 100 Euros we paid as bond, but if you think it can't come worse, so did I. The universe always finds new ways to catch us off guard. The owner also had a Porsche, of course, which he very, very cleverly parked right in front of the party location. It was one of those free, not secured parking lots and in the next morning he saw that someone had stuffed an empty pack of cigarettes into his exhaust.

I don't know who did that, or whether the person was invited to the party, or had been to that party, but the owner was pretty omniscient about everything - not that he had actually seen anything but he "knew". Well, it seems you gotta know your rights very well in this land, else you're fucked. He threatened to beat me up, if I don't remove that pack of cigarettes from his exhaust pipe immediately and told me that he'd send the car to the Porsche shop and have the mechanics look all over it and send us the bill for it regardless whether something was broken or not. A nice gentleman. Especially if common sense tells you that a Porsche engine probably could blow out anything that is in the exhaust pipe with such a speed that it could kill a cat ~~or baby~~. But ok, nothing against that. We apologized to everyone for everything and weather and bought some nice please-forgive-and-forget-us gifts and hoped that it would be over now. Last we asked someone at the Red Cross, they told us that the Porsche was gone for a couple of days...

--
Black

Some News & Stuff

Andreas 'BlackHC' Kirsch — Fri, 15 Sep 2006 12:29:26 +0000

Howdy,
it's been a while since my last post and to sum what has happened since then: nothing. Well that may not be entirely correct but it fits pretty much. I've heard that SplashDamage is looking for a game programmer with no required CS degree, so I thought: hey, why shouldn't I simply apply for the fun of it.

As it turns out, writing all your application texts (most for the first time) in English isn't that entertaining. To put it nicely, it's dead boring. So I'm stuck with a finished CV and a few prepared source codes and don't know when I'll get over my laziness and write the rest. Moreover, I want to finish my C++ raytracer first (at least get the design nicely together), so that I can add it to my portfolio.

And since I'm still pretty busy working on Shelled and fighting Don Quijoteish battles in my real-life, I don't believe that anything will ever be finished anytime soon.

Anyway, I have dug out some really old renders from my raytracer and they look neat - even if the results were neither wanted nor expected...

Some beautiful rendering

This was taken after putting in some neat refraction and reflection code, but since no energy was lost during these operations, it looks like some weird mirror thing. Anyway, neat, isn't it?

Regarding Shelled! everything is going great and it's a really fun game and I can hardly stop playing once started.

Stay tuned for updates regarding my raytracer and Shelled! and enjoy life like my fish does,
Andreas

My Visit of the Games Convention 2006

Andreas 'BlackHC' Kirsch — Mon, 28 Aug 2006 19:15:26 +0000

The Games Convention: Europe's biggest game show - and I was there! Rather I was one of 183,000 visitors, so you can imagine how big this event is and how damn lost you can get inside the big convention halls.

It was fun nevertheless and a great thing to see. There were DJs, special guests, moderated events and and and everywhere all the time. And I better not forget to mention all the gorgeous booth babes that were around.

Yeah, GC was really a visual feast. But first things first: A friend and I arrived at about 11 am and stayed for 6.5 long hours and visited every hall and almost every booth. Some of our highlights were:

Saw some great case moddings (pictures below)
Saw Sebastian Lahm (German soccer player who is in the national team) at the Nintendo booth signing footballs
Heard some known hip-hopper singing at the EA booth (but silly me, I've already forgotten his hip-hop artist name)
Played and watched a preview of Crysis
Previewed Gothic 3 and watched the trailer
Played Stalker - yeah, it's finally going to be release soon - really!
Met Fatal1ty at the Creative Labs booth, where he moderated a lottery. People had to ask him good questions to win a prize -_ but I didn't dare ask him about the reasons why he lost against the Chinese Doom3 player, should I have?_
Stayed in queue for about 30 min to play Enemy Territ ory: Quake Wars (after other friends of mine had waited for an undisclosed time but gave up their seats for us - thanks to them, Michi & Olli). It really kicks ass and I can't wait to play it online.
We also met Wils and Digibob at the SplashDamage/ET:QW booth (bongoboy was around, too, but I didn't know him from IRC). Wils was so kind to guide me around in ET:QW and show me some tricks throughout the game session. Thanks Wils!
Saw a lot of hot chicks but I've mentioned that already, didn't I?

Ok, ok, so this was the text part. I know that you all want to see some pictures, so here we go - but due to their amount and the lack of auto-generated thumbnails (thank you, WordPress), I had to add a 'More' tag. I hope it doesn't bother you. Enjoy the pictures.

Now, let's start with the case modding pictures. They aren't great and there aren't many of them since I needed the flash space for something else - you guess.

We also saw some guys dancing on dance pads, the nice booth of EVE online and a live reggea band playing some good music:

There were also like 30 plastic models of famous game and comic characters (Solid Snake, Spiderman, Ms. Croft, and and and). Most of my pictures only look good at 100x100 since they are too blurry otherwise, so I'm only going to show you a famous orc and the batman.

Another cool thing, which hadn't had anything directly to do with games, were some hot cars (notice the booth babes behind the orange one - they were hot, too):

At the Midway stand (publisher for Unreal Tournament 2007, etc) there was a huge - and with huge I really mean it - Unreal Tournament model. It was like 4 meters high and looked like rendered - my fish points out that things are normally the other way but it really looked like it was taken out of some UE3 screenshot..

And now finally we arrive at the type of pictures you have all waited for: the booth babes.

Let's see: do you like Lara Croft?

Okay, that shot wasn't perfect, so what do you think of this one?

A better shot of Lara Croft

Better eh? Or if you don't like her, what about her?

Maybe you like them in conjunction with cars?

Or do you want to see them in a group?

I hope you enjoyed these photos as much as I did when I took them..

Stay tuned,
Andreas

The Return of QuickGamma

Andreas 'BlackHC' Kirsch — Tue, 22 Aug 2006 12:21:16 +0000

QuickGamma is a very nice tool, originally written by Randy 'ydnar' Reddig, which allows you to set and reset the screen's gamma easily. The program itself stays nicely hidden in the tasktray, so it won't take away any valuable taskbar space.

You can download the new 0.1.2 version (which was created by me based on ydnar's 0.1.1 release) here.

After my explorer crashed horribly a few times yesterday and I had to kill the old QuickGamma app in the task manager and then restart it again each time, I decided that something had to be done about it.

So after searching for too long in MSDN and with Google, I finally found some documentation on some custom-registered "TaskbarCreated" message. Shame on you, Microsoft, for not making this information easily accessible!

Anyway, I've added support for automatically reregistering the QuickGamma tray icon after an explorer restart and also removed the idle thread priority, so it should always be responsive from now.

Stay tuned and enjoy the new version of this neat, little tool.

Andreas

PS: It also seems that the old download link is down, so, if you want the original QuickGamma, simply ask me.

Sploidz - About a Game

Andreas 'BlackHC' Kirsch — Mon, 21 Aug 2006 17:01:48 +0000

Finally after 8 months of hard but fun work Sploidz is done and has been released by Red Thumb Games. I was the lead (and only) programmer for that project (David House did code the prototype, however) and I'm so proud to see it released and played by you!

Here check out this screen:

Weee, exciting, isn't it?

If you want to find out more about the project, don't forget to check it out in the project area or download it directly from http://www.sploidz.com.

Enjoy the game,

Andreas

Everybody can use GLSL - even I.

Andreas 'BlackHC' Kirsch — Tue, 15 Aug 2006 00:27:27 +0000

Well, it's hard to believe but true. Yesss, I seem to have mastered the basics of GLSL and I've really managed to write a phong shader that even supports normal maps and tangent space.

You validly wonder how I achieved this? The answer is..

Typhoon Lab's Shader Designer!

It's really a great tool to quickly develop GLSL shaders and preview them with various textures and models. It also includes all necessary OpenGL extension specifications and the GLSL specs. You haven't downloaded it already? Go get it, or maybe not? Perhaps I've been paid by them to do some illegal ~~(or is this kind of obvious surreptitious advertising legal?)~~ advertising? - my fish now scowls at me; you wouldn't believe that fish can do that.. No, to be honest I'm just very excited about GLSL and everything. And this tool has certainly helped me debug my shaders and gain some experience with the language before taking on some serious project.

Before I forget it, I've uploaded my first (and right now only) Shader Designer project for you. The one that supports tangent space - yay!

Here, grab it: First ShaderDesigner Project

But now I'll tell you the actual reason why I'm so excited: I've added GLSL shader support to Vengeance2. You might have to wait for a new build that supports the feature though, since it is brand-new - you could, however, directly grab the SVN version and compile it yourself to be able to see these hot effects now. OK, there are dozens if not hundreds of other engines which already support phong shading and probably do so better than my code does but hey, do you remember - I'm excited!

About the dev aspect of it:

It's been incredibly easy to add GLSL lighting support to this Quake1 engine. At first I doubted that I would be able to add support for lighting the worldmodel (the map), too, but it turned out to be pretty easily and it even is relatively fast. I'm using distance checks and PVS culling, which both turned out to be very easy to code and yield good performance. Thus I'm quite happy with the result.

For the ones of you, who always want to see some hot pics and some action, I've got a few screenshots for you:

Stay tuned and remember the wise words of my fish.

Andreas

Hello, my Name is..

Andreas 'BlackHC' Kirsch — Mon, 14 Aug 2006 22:50:58 +0000

This is my first post here and I guess people introduce themselves with their first post. I'm going to introduce my fish instead. He's so much cooler than I am - although I'm bigger than him, but who isn't?

You see, actually I don't even know whether he really is a 'he', if you understand what I want to say. I mean, how do you determine this for a fish? I guess the male fish don't have a, well, a counterpart to what the female fish don't have.. you understand, do you?

Actually I think: I shouldn't know. People who know such stuff and publicly tell other people about it.. Who cares anyway? It's a fish after all. Though.. no, no, let me go back to the task of describing the fish - my fish.

It - let's call it it for a while although now he it looks at me a bit furiously - isn't that big, a lot smaller than my 1.88m. I daresay as big as fish nowadays get but you never know after Chernobyl and all that other almost melt-downs and leaks. The color of fish is indifferent, isn't it? Let's just say my fish is yellow. I don't want to discriminate my fish. With all that lawsuits because of people's color and race and smoking habits, I really don't feel like taking part in that discussion.

I don't want to hurt anybody's feelings. So let me simply declare my fish is some yellow fish. It's a pity that it - you can't say no to those eyes - he has no arms and no hands but only fins. We're already a perfect team but we would be invincible, if he could give me a hand with coding. But the way things are, he can only assist me with his mind and his well-educated comments and notes on my work. You're going to appreciate his words, too - believe me.

By the way if someone wants to know more about me, simply take a look at the About page.

Andreas

BlackHC's Adventures in the Dev World

Reading the Deep Learning Book - Chapter 2

Moore-Penrose pseudoinverse

Aside: there is also a right pseudo-inverse

Why can we invert \(AA^T+ \alpha I\) for positive \(\alpha\)?

Can \(A^TA\) have negative eigenvalues?

What if \(A^T A\) was invertible?

Two properties of the pseudoinverse

A related optimization problem

Broadcasting notation: oh my...

Looking back and ahead

Recent Medium Posts

Markdown Blog

Writer's block

Imitating PBRT-style literate programming in LaTeX

Creative learning, Scratch and alternatives for learning how to create games

Al et WML

WML - Whitespace Markup Language

Afterthoughts

Unity Prototype Project

Presentation (seminar) about convex analysis

Molecular Dynamics and CUDA---my interdisciplinary project

What is molecular dynamics?

What is MarDyn?

Previous work

The main aims of the project

Chronology of my work

What I've done

Final paper & code

Alternatives

A Long Journey: Acceler8 and TBB

Reading Nonfiction Books Quickly

Speed Reading

About Reading in General

Own Expierence

Presentation Advice

T61 Extractor

Books on Numerical Analysis

ANTLR Stupidity (Warning 209)

Limbo

Light Propagation Volumes

Panorama Stitching

Rotation of Low Order Spherical Harmonics

Rigid Body Motion

PowerPoint LaTeX

A quick note on quantifiers: For almost all and there exist infinitely many

Random Things

Prototype

Resident Evil

Red Faction: Guerrilla

Brüno

Shadow Complex

Summing Formulas

Extracting Information from StudiVZ

My Bachelor Thesis

Post-Mortem

Preparations

Writing the Thesis

Sploidz Revisited (Unofficially)

Development

Interesting things to know

More Book Reviews and re: Design Patterns

Pattern-Oriented Software Architecture - A System of Patterns (PoSA I)

Architectural Patterns

Design Patterns

Idioms

PowerPointLaTeX Update

Implementation Note

Semi-Conductor Optimization (Uni Project)

From Vertex Positions to Packed Element Arrays

Using the Input-Assembler Stage without Buffers (OpenGL)

Packing Vertex Positions into the Elements Array

Fast Forward

Movies and Games

Memento

Fahrenheit

Books

The Pragmatic Programmer

OpenGL Superbible

Multi-Core Programming