Hot answers tagged web
11
From the Surfraw website:
Surfraw provides a fast unix command line interface to a variety of popular WWW search engines and other artifacts of power. It reclaims google, altavista, babelfish, dejanews, freshmeat, research index, slashdot and many others from the false-prophet, pox-infested heathen lands of html-forms, placing these wonders where they ...
11
If you're looking for an online environment to practice your pattern matching using regex, there are a number of nice resources that allow you to play around with expressions for a given portion of text. Off the top of my head, there is: regex pal or rubular.
In terms of exercises, one idea is to search for questions tagged with [regex] on stack overflow. ...
9
Rendering HTML is a function of the browser, not the operating system. Don't let Microsoft's patently ridiculous marketing of "native support" delude you into thinking otherwise.
Install a modern browser.
Live happily ever after.
8
The difference is in how DNS and the HTTP "Host" header work.
The site you're going to may have multiple sites hosted using the same server. In DNS, all the names for the sites hosted on that IP list the single server IP address. When you enter the name in the browser, the browser sends the hostname to the server using the "Host" header. If you enter only ...
7
No, it will not help. All it does is set up an encrypted connection to your local machine, and from there, connects to the outside world exactly like it would have without the local ssh - you gain nothing, but performance is going to suffer (after all, ssh does encrypt and decrypt all the X11 messages that firefox and the X11 server pass back and forth).
...
7
For simple cases of downloading the contents of a page, use curl or wget. Both are command line tools designed to download files over HTTP and have many options. In your case, you'll likely need to make these tools look more like a browser; lutzky's answer and penguin359's answer mention some curl and wget options that are useful in that respect.
Sometimes, ...
7
Yes, it's fully possible with curl. Most importantly will be to save and reload cookies between uses of curl with --cookie-jar. You can also post form data as needed. I usually use a Firefox add-on called Live HTTP Headers to capture what happens when I navigate a website. It will record any headers, but also any form posts which is very helpful when ...
5
Note: In your case, the best would be to just drop root privileges for updates and run your scripts with your apache user:
su apache -c "./update-script"
Otherwise, use chmod g+s /var/www. New files and sub-directories created inside this directory will share the same owner/group as the parent directory, by default. (This spreads recursively.)
...
5
I would suggest that uzbl is just the right ninja magic for this. It is a scriptable, console-controllable single-purpose browser. Being based on webkit, its rendering and javascript support is first class, but it follows the unix phylosophy of doing one thing and doing it well while allowing other programs to push data in and out.
There is a wrapper for it ...
5
LinkChecker.
LinkChecker is a free, GPL licensed URL validator.
Features
recursive and multithreaded checking
output in colored or normal text, HTML, SQL, CSV, XML or a sitemap graph in different formats
HTTP/1.1, HTTPS, FTP, mailto:, news:, nntp:, Telnet and local file links support
restriction of link checking with regular ...
4
Gwibber has the ability to send to multiple services at once. According to their website it supports the following protocols/services:
Twitter
Identi.ca/StatusNet
Ping.fm
Facebook
FriendFeed
Buzz
Digg
Flickr
Qaiku
As far as I know, it has the ability to receive content from all of the listed services, but I'm not sure if there is a way to receive ...
3
I think you are looking for a caching web proxy like polipo or squid. Although it's called a proxy-server you don't need a separate system to run it. It just means that it is a special software that can serve a client a particular service (In this case caching already downloaded data). A client is also a software that can run on the same system as the server ...
3
Yes, of course. You need X11 installed but that is all. You could use something like fvwm, or awesome or xnomad to manage the X11 session. Configuration of those window managers can be done via their own rc files and started via .xinitrc/.xsession. Then from a shell you can start either Chrome or Firefox.
If you do not want to install X11, then you can ...
3
It does not rely on the desktop environment. What it does rely on (at least in case of graphical-mode browsers like Chrome, Opera or Firefox) is the Xorg server, together with its libraries1.
Once you have the core dependencies of a browser installed (including the X server), you can run it under a minimal environment either by:
using a very minimalistic ...
3
You can try links or lynx. Both should be available through apt-get. These are text only browsers, and usually get the job done relatively well when all you have is a console. I don't think any console browser has PDF support though, since that format is usually highly graphical in nature.
3
"Stealing cookies from the web browser" in order to obtain authentication sounds like exactly the kind of thing malware would get up to in order to gain access to your personal information and login data.
If this approach turns out to work, I recommend you inform each browser's development team about a security hole in their software.
3
Answer
No, it is not useful. If your goal is to protect your data in transit over the WiFi connection, then an encrypted tunnel over your loopback interface does not serve that goal.
Research Hints
This is not a comprehensive list, but it will get you started as you search for a good solution for protecting your data in transit.
SSL
TLS
IPSec
SSH SOCKS ...
3
My favorite is wget, so I'll give an example with that. What you want to do is replicate your browser session as closely as possible, so use the relevant commandline arguments. The ones which are necessary depend on how thoroughly the site checks your browser. --referer (sic) is usually sufficient, but you might also need --user-agent and --load-cookies.
...
3
A lot of excellent web UIs are mentioned in other answers, however if you are managing VMs, I would recommend using OpenStack. Just for UI, it'll be overkill but trust me requirements are always increasing.
OpenStack will provide you an API upon which you can build an interface easily both for customers and the administrators and modify to suit specific ...
3
I would suggest xen-admin , it has the capabilities to give your users the ability to mange their VM.
And maybe Proxmox is good too, but i don't have any experience with it.
3
Not having root privileges prevents you from listening on ports below 1024 on typical Linux systems.
Thus, Squid should work OK as non-root listening on 8080 as long as the system you are on hasn't blocked incoming traffic on that port via a firewall or iptables.
Not sure how many file descriptors Squid uses typically but if your admin has set a limit for ...
3
I don't know if there's online resources, but local tests is fine, e.g with kiki
This will help you learn about regex.
The package is available in Ubuntu / Arch repository, to install it, do:
apt-get install -y kiki
Or pacman -S kiki-re on Arch.
3
You can give http/rtsp/rtmp input to ffmpeg and it would compress it. But the catch is you need the actual video url.
For sites like youtube, dailymotion, the webpage address is different from the video address.
So you need a helper program like youtube-dl or Movgrab which can provide the video url as well as download the video.
Then you can use ffmpeg to ...
2
well, opengrok for opensolaris code base but no opengrok for linux codebase, one could start one however but the cost involved in setting up and maintaining is too greater I suppose. kernel.org maintains only kernel source tarballs. It has been left to the individual distro maker to distribute source code along with the product OS but practices are seldom ...
2
A browser usually sends a list of accepted languages with every request. CMSes often use this when no cookie is set. (In theory, the cookie is just a fallback and the language settings are correct.) This results in <span>Hello Guest</span> for me:
curl -s --location -H 'Accept-Language: en' ...
2
webmin, THE tool, it doesn't have all the features you have mentioned, but has a lot more.
Still it doesn't intended for the new user of linux, since you should have some knowledge about the meaneing of the services, permission and so on, but it give you a nice GUI front end to work with, and don't bother with text editors and so on.
Since Linux is not ...
2
Stealing cookies from a web browser is a standard thing when you're writing a scraping script and don't want to deal with some complex website authentication in your script.
Traditionally, browsers stored their cookies in a simple text format inherited by Netscape, the cookies.txt format. Wget has an option --load-cookies to load a file like this.
Modern ...
2
You can't do this in a sane way for Google search queries.
Why? Google is using HTTPS (SSL) everywhere now. All traffic between the endpoints (browser & Google server) is encrypted and checked for integrity.
The only way to read (let alone modify) encrypted traffic is changing it at or beyond on one of the endpoints. Suggestions in the direction of ...
2
openssl s_client -CAfile /etc/ssl/certs/ca-certificates.crt -verify 12 -showcerts -connect mail.google.com:443
That will work on debian or its derivatives where the list of trusted CAs (of the system which may not be the same as Firefox's) is in /etc/ssl/certs/ca-certificates.crt. You may need to adapt on other Unices.
Note that it will not query ...
2
Just a suggestion that satisfies the online and regex portion of your question:
http://gskinner.com/RegExr/ is a pretty friendly regex testing tool. It highlights results on the fly as you create your regex pattern. I use it regularly (there's also a desktop version using Adobe AIR)
As tojrobinson said, there's only a subset of regex implemented in grep, ...
Only top voted, non community-wiki answers of a minimum length are eligible
