Hi there 👋

[03.14] Growing inequality is very concerning Wealth inequality is going to the extreme. In the US, the top 1% of wealthiest individuals hold around 32–35% of total wealth. The top 10% hold around 80% of total wealth. The rest of the society is being cornered to the very bottom. You see, salary growth really dwarfed in comparison to asset price growth, e.g. stocks and real estates. For every 3x salary increase, there’s at least 10x increase in asset prices....

This is a reference for physics at the level of AP physics or college physics that is typically taught during the first year of university. It is mostly taken from the widely used textbook “university physics with modern physics”. ( Citation: Young, Freedman & al., 2011 Young, H., Freedman, R. & Ford, A. (2011). University physics with modern physics 13th edition. Addison-Wesley Reading, MA. ) Understanding of physics is very important because our entire society is built upon it....

Basics An option is a financial contract that gives you the right (but not obligation) to buy or sell usually 100 shares of the underlying asset at a strike price $X$ at or before some specific date, usually Fridays. The price of an option is quoted on a per share basis, for example 5.5 USD. To buy the option, you have to pay 5.5 x 100 = 550 USD. If the option later rises to 6....

At the same time LLMs are changing the world, we have witnessed fast progress in AI image and video generation. AI models are now able to synthesis high quality, high fidelity and high resolution images via text prompting. Mainstream generative model architectures have shifted from VAEs, flows and GANs to diffusions and transformers. In this post I take notes of several vision models. It may be regularly updated to reflect latest research development....

In this post I will walk through the transformer layer and several improvements over this architecture that are commonly employed in many popular open source large language models (LLMs) today, for example Llama. Discussed include SwiGLU and RMSNorm layers, RoPE and ALiBi position embeddings; and finally Flash Attention for scaling attention calculation to long sequences. We will use Llama source code as example implementation, and toward the end I’ll go through the rest of Llama’s source code....