You can delete most lines or shapes from a drawing by simply selecting them and then hitting the Delete or Backspace key.
You can also right-click on a line or shape to bring up a right-click menu with the Delete option:
Please note that certain objects in various diagrams are not deletable by design. For example:
In Mind Maps, if you try to select everything:
And delete, you are left with the Main Topic as it is the starting point of every Mind Map:
Gantt Charts are another example. If you select everything:
And delete, you are still left with the Gantt Chart table without any tasks: