1. Sink and Source
Stuff disappears into a Sink;
Stuff comes out of a source.

"Stuff" could be Voltage (a Voltage Source), Current (a current source), Heat (a Heat sink), etc, etc, etc...

"If 'I sink a transistor' - it's base (can I say so?)"

No. That would be meaningless.

2. Pull-Up
This has been covered before - try a 'Search' for "pull-up", "pullup", and "pull up"

"If I put a 'regular' LED on AT89S51's P0, will it 'pull up' the current"

Not exactly: "Pull-Up" refers to voltage, not current.

"and then acting as a resistor"

Definitely not!
This is why you usually need a resistor in series with a LED: it will not limit the current at all; it will take as much current as can possibly flow in the circuit - leading to its own destruction, and possibly also other parts of the circuit.

Note, however, that you can buy "constant-current" LEDs, and LEDs with built-in series resistors - these would not need any additional external current-limiting arrangements (within the specified limits)

"According to the electronic specs is looks like a bad idea to drive a led from the pins directly"

If that's what the specs say, then that's what you must do (or not do).

"Is a transistor the proper solution?"

In many cases, it is fine to connect LEDs to port pins with suitable series resistors (or other current-limiting arrangement)

One thing to beware of is that the MCU may not be able to drive LEDs on all its port pins - check the datasheet

3. "voltage regulator" vs "operation amplifier"

A Voltage Regulator is a device which gives you a constant output voltage despite changes in input voltage and/or load current;
An operational amplifier ("Op-Amp") is just an amplifier - you might well use one as part of a voltage regulator circuit!

4. step up/down

Step Up: the output voltage is greater than the input (needs switch-mode);
Step Down: the output voltage is less than the input (can be linear or switch-mode).