Hi there 👋

[04.10] Tariff move is necessary to save America The US buys goods from around the world with US dollars. Then with the dollar as a world reserve currency held by so many countries, it is so tempting for the US government to borrow that money. It’s almost like printing money, except that you have to pay interests back later. That’s how the US has accumulated massive amount of debts that is non-sustainable....

This is a reference for physics at the level of AP physics or college physics that is typically taught during the first year of university. It is mostly taken from the widely used textbook “university physics with modern physics”. ( Citation: Young, Freedman & al., 2011 Young, H., Freedman, R. & Ford, A. (2011). University physics with modern physics 13th edition. Addison-Wesley Reading, MA. ) Understanding of physics is very important because our entire society is built upon it....

Basics An option is a financial contract that gives you the right (but not obligation) to buy or sell usually 100 shares of the underlying asset at a strike price $X$ at or before some specific date, usually Fridays. The price of an option is quoted on a per share basis, for example 5.5 USD. To buy the option, you have to pay 5.5 x 100 = 550 USD. If the option later rises to 6....

At the same time LLMs are changing the world, we have witnessed fast progress in AI image and video generation. AI models are now able to synthesis high quality, high fidelity and high resolution images via text prompting. Mainstream generative model architectures have shifted from VAEs, flows and GANs to diffusions and transformers. In this post I take notes of several vision models. It may be regularly updated to reflect latest research development....

In this post I will walk through the transformer layer and several improvements over this architecture that are commonly employed in many popular open source large language models (LLMs) today, for example Llama. Discussed include SwiGLU and RMSNorm layers, RoPE and ALiBi position embeddings; and finally Flash Attention for scaling attention calculation to long sequences. We will use Llama source code as example implementation, and toward the end I’ll go through the rest of Llama’s source code....