Half a croissant, on a plate, with a sign in front of it saying '50c'
h a l f b a k e r y
The best idea since raw toast.

idea: add, search, annotate, link, view, overview, recent, by name, random

meta: news, help, about, links, report a problem

account: browse anonymously, or get an account and write.

user:
pass:
register,


                                     

Low-pass filter for news

Filter out the stories which don't say anything new
  (+4)
(+4)
  [vote for,
against]

Take an on-line news service like CNN or the New York Times. Scan through each day's stories for all non-stop-words, and for each keyword compute the fraction of stories in which it has occured over the last T days (T being a user-set parameter). Then create a table of contents of the day's stories, listing beside each title their Shannon information content, approximated as though each keyword was an independent random variable. Stories whose keywords have been very common for the last T days presumably tell you little that's new, so they get a low score. List stories in order of decreasing score, if you like.

Keep separate frequency-databases for different news-sources you read, or amalgamate them if they cover similar topics.

This procedure will over-estimate the information content. A more subtle algorithm would induce which words cluster together, e.g. {"Watergate", "Nixon", "Haldeman", "plumbers", "18 and 1/2"} or {Lewinsky, Starr, stain}, and treat those clusters as its independent variables. The statistical language processing wallahs already have such inductive algorithms.

cosma, Jan 28 2000

Term weighting widely used for text retrieval http://www.ftp.cl.c...text-retrieval.html
This is called "term-frequency weighting" and is in common use by the search engines. Much to my disgust, I found that information-theoretic weights don't work nearly as well as some ad-hoc methods. See the paper "Simple, proven approaches to text retrieval." [rmutt, Jan 28 2000, last modified Oct 04 2004]

Etxtreme http://www.etxtreme.ru
This seems to work with this general principle, but as applied to a huge group of very diverse URLs instead of a TV news program. [monde, Jan 28 2000, last modified Oct 04 2004]

Teefax..ohhh those blocky, limited palette colours... https://twitter.com...tag/Teefax?src=hash
[not_morrison_rm, Sep 25 2016]

[link]






       Isn't this really a high-pass filter?
egnor, Mar 02 2000
  

       High and low in "X pass filter" refers to the frequencies that are allowed through the filter.   

       What I think egnor correctly observes is that the filter cosma describes prefers drastic, fast change (as in the amplitude of a high-frequency signal) over slow change (as in the amplitude of a low-frequency signal.)
jutta, Oct 25 2000
  

       The "Etxtreme" link is broken for me. Did the site go away, or is it temporarily down, or is the URL incorrect?
egnor, Oct 26 2000
  

       These days this would result in a blank tv screen.
RayfordSteele, Aug 14 2007
  

       What's needed is a news site with a user-configurable view.   

       With a regular news site, links on the front page to business, politics, entertainment, arts, science, sport ... below that, more choices and so on.   

       A configuration page could allow a user to select-   

       + Science
:+ Geology
:: + Earthquakes
:: + Volcanos
:: - Floods
:: - Avalanches

or

+ Sport
: + Ball Sports
:: +Team Sports
::: + Basketball
::: - Football
::: + Hockey
::: - Rugby
:: + Individual
::: + Golf
::: - Tennis

The page(s) would then only display items of interest.
  

       Oddly, this doesn't seem to exist anywhere.
8th of 7, Sep 24 2016
  

       Yep, this is definitely a high pass filter, where most stuff comes out as grey with no information content, except when something new happens. Depending on the Q of the filter, it may result in a peak toward maximum information until the information becomes old hat, at which point it may 'ring' and deviate back through grey to a null information point, where no information about the relatively slightly old new news occurs (probably by negation - denying that it ever existed). This would give rise to an unsharp masking effect, sharpening the news.
Ian Tindale, Sep 24 2016
  

       //Oddly, this doesn't seem to exist anywhere.// But it used to - Ceefax, Oracle, Prestel.
MaxwellBuchanan, Sep 25 2016
  

       Hmm, somebody is tinkering with a new Ceefax service called Teefax, not having a raspberry pi, or a tv and being in the wrong country, I'll have to take their word for it. See link
not_morrison_rm, Sep 25 2016
  

       The answer would seem to be an application that is gven a list of categories and a list of URLs and then searches and collates the results, then displays the results as a web page.
8th of 7, Sep 25 2016
  

       //The answer would seem to be an application that is gven a list of categories and a list of URLs and then searches and collates the results//   

       Haystack does that quite well, monitoring tags and categories.   

       But for those interested in Shannon Doherty dating Mike Nyquist, following a developing news story becomes complicated. Sometimes things escalate and a new RSS headline is warranted. Sometimes you don't care if there's severe weather the other side of the planet, but want to know if its bearing down on you.   

       The next question is - Do we need dedicated hardware for news feeds in preparation for November 8th ?   

       Trump's son apparently thinks that the monkey that might get elected will do just fine as its well supervised. Since when did a president need adult supervision ? And why does supervision now sound like a superhero skill ?
bigsleep, Sep 25 2016
  

       // Since when did a president need adult supervision ? //   

       No. 15, James Buchanan, widely regarded as the worst president ever, and that's against some seriously formidable opposition...   

       (And yes, definitely the same lineage as the erstwhile [MB] of halfbakery reknown, and in the same fine aristocratic tradition of "Public service by leadership, usually off the edge of the nearest precipice").   

       //And why does supervision now sound like a superhero skill ? //   

       Well, would you take the job ?
8th of 7, Sep 25 2016
  

       //James Buchanan// She was definitely the black sheep of the family.
MaxwellBuchanan, Sep 25 2016
  

       // She was definitely the black sheep of the family.//   

       Why not just say something genuinely self-indulgent rather than churn out the same repetitive attention seeking narrative ?
bigsleep, Sep 25 2016
  

       How soon should before election day should I start shorting the market, buying ammunition, and cornering the market on nitrogen and other consumables?
RayfordSteele, Sep 25 2016
  

       "Canned food and shotguns ..."   

       Probably best to start right now. Stock up on Mexicans too, there's like to be a shortage ...
8th of 7, Sep 26 2016
  
      
[annotate]
  


 

back: main index

business  computer  culture  fashion  food  halfbakery  home  other  product  public  science  sport  vehicle